2024-09-13T13:12:18.132 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T13:12:18.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T13:12:18.271 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647 branch: wip-guits-main-2024-09-13-0840 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7903647' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-13-0840 ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - CEPHADM_STRAY_DAEMON log-only-match: - CEPHADM_ sha1: b0201b8c79733293453e7f10a10c7fa43119222b ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: b0201b8c79733293453e7f10a10c7fa43119222b selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-main-2024-09-13-0840 sha1: b0201b8c79733293453e7f10a10c7fa43119222b owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4031 sha1: b0201b8c79733293453e7f10a10c7fa43119222b sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-13-0840 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_b0201b8c79733293453e7f10a10c7fa43119222b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: b0201b8c79733293453e7f10a10c7fa43119222b targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJEmEYdLrAzt6SSfUN9jpRt/MOPkAAhz3xE7UweSMr9MxsxTYOJ1pcMLEUY4of+3GIb4Ibqr0oi3vMbe/ZzPkA= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIqK6MBO4YPcb86PTNvi+7NOlkSsltnDwsG27wmfILdKVungwkj2n6mOumugE2mtaRYhKW4Ex/ncoNs49wQ0VSU= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_12:30:34 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T13:12:18.271 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_b0201b8c79733293453e7f10a10c7fa43119222b/qa; will attempt to use it 2024-09-13T13:12:18.272 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_b0201b8c79733293453e7f10a10c7fa43119222b/qa/tasks 2024-09-13T13:12:18.272 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T13:12:18.274 INFO:teuthology.task.internal:Checking packages... 2024-09-13T13:12:18.296 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b0201b8c79733293453e7f10a10c7fa43119222b' 2024-09-13T13:12:18.297 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T13:12:18.297 INFO:teuthology.packaging:ref: None 2024-09-13T13:12:18.297 INFO:teuthology.packaging:tag: None 2024-09-13T13:12:18.297 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-13-0840 2024-09-13T13:12:18.297 INFO:teuthology.packaging:sha1: b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:12:18.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-13-0840 2024-09-13T13:12:18.450 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4917.gb0201b8c 2024-09-13T13:12:18.451 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T13:12:18.458 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T13:12:18.458 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T13:12:18.471 INFO:teuthology.task.internal:Saving configuration 2024-09-13T13:12:18.482 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T13:12:18.491 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T13:12:18.524 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 13:07:04.457179', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJEmEYdLrAzt6SSfUN9jpRt/MOPkAAhz3xE7UweSMr9MxsxTYOJ1pcMLEUY4of+3GIb4Ibqr0oi3vMbe/ZzPkA='} 2024-09-13T13:12:18.558 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 13:07:04.456096', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIqK6MBO4YPcb86PTNvi+7NOlkSsltnDwsG27wmfILdKVungwkj2n6mOumugE2mtaRYhKW4Ex/ncoNs49wQ0VSU='} 2024-09-13T13:12:18.558 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T13:12:18.565 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-13T13:12:18.565 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-13T13:12:18.565 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T13:12:18.652 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fefe8f83be0>, signals=[15]) 2024-09-13T13:12:18.652 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T13:12:18.659 INFO:teuthology.task.internal:Opening connections... 2024-09-13T13:12:18.659 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T13:12:18.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:12:18.749 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-13T13:12:18.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:12:18.834 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T13:12:18.842 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-13T13:12:18.861 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-13T13:12:18.861 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-13T13:12:18.918 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-13T13:12:18.918 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-13T13:12:18.918 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-13T13:12:18.918 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-13T13:12:18.918 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T13:12:18.919 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T13:12:18.920 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-13T13:12:18.953 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-13T13:12:18.971 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-13T13:12:18.972 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-13T13:12:19.029 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-09-13T13:12:19.029 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-09-13T13:12:19.029 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T13:12:19.030 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T13:12:19.031 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-13T13:12:19.061 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T13:12:19.071 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T13:12:19.078 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T13:12:19.078 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-13T13:12:19.081 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-13T13:12:19.098 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T13:12:19.107 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T13:12:19.107 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-13T13:12:19.139 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-13T13:12:19.227 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T13:12:19.304 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T13:12:19.320 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T13:12:19.320 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T13:12:19.320 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T13:12:19.321 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T13:12:19.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:12:19.337 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-13T13:12:19.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:12:19.339 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-13T13:12:19.395 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-503.el9.x86_64 2024-09-13T13:12:19.395 INFO:teuthology.task.kernel:Running kernel on smithi189: 5.14.0-503.el9.x86_64 2024-09-13T13:12:19.395 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum install -y kernel 2024-09-13T13:12:19.397 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-13T13:12:19.397 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-503.el9.x86_64 2024-09-13T13:12:19.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-09-13T13:12:21.945 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-13T13:12:22.813 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-13T13:12:27.687 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-13T13:12:27.730 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 20 MB 00:03 2024-09-13T13:12:32.229 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-13T13:12:32.547 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.5 MB 00:00 2024-09-13T13:12:33.996 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-13T13:12:34.306 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-13T13:12:35.650 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-13T13:12:35.693 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-13T13:12:41.357 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-13T13:12:41.415 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-13T13:12:43.371 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T13:12:43.371 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T13:12:43.389 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T13:12:43.389 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T13:12:43.459 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:12:43.463 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:12:43.463 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-13T13:12:43.463 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:12:43.463 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:12:43.463 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-13T13:12:43.464 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:12:43.465 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 79 M 2024-09-13T13:12:43.465 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 126 M 2024-09-13T13:12:43.465 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:12:43.486 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T13:12:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T13:12:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T13:12:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:12:43.492 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:12:43.492 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:12:43.492 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-09-13T13:12:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:12:43.492 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 79 M 2024-09-13T13:12:43.493 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 126 M 2024-09-13T13:12:43.493 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:12:43.658 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 253 kB/s | 23 kB 00:00 2024-09-13T13:12:43.859 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 259 kB/s | 23 kB 00:00 2024-09-13T13:12:44.066 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-13T13:12:44.458 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 36 MB/s | 29 MB 00:00 2024-09-13T13:12:44.792 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-13T13:12:44.792 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:12:44.792 INFO:teuthology.orchestra.run.smithi043.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-13T13:12:45.506 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:12:45.892 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:12:45.892 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:12:46.009 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-09-13T13:12:46.932 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:12:46.932 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:12:47.517 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.8 MB/s | 29 MB 00:03 2024-09-13T13:12:49.226 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.3 MB/s | 35 MB 00:05 2024-09-13T13:12:49.226 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:12:49.226 INFO:teuthology.orchestra.run.smithi189.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-13T13:12:49.503 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:12:49.910 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:12:50.181 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T13:12:50.272 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:12:50.272 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:12:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T13:12:51.249 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:12:51.249 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:12:53.075 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T13:12:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T13:12:53.698 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:12:54.359 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T13:12:54.443 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T13:12:57.251 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T13:12:57.380 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T13:12:58.322 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T13:12:58.371 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:02.212 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T13:13:02.330 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:07.151 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:42.568 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:42.957 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T13:13:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T13:13:42.958 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T13:13:43.795 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:43.795 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:13:43.795 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:13:43.795 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T13:13:43.795 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T13:13:43.795 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T13:13:43.796 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T13:13:43.796 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:13:43.796 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:13:44.153 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T13:13:44.247 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T13:13:44.247 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T13:13:44.248 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T13:13:44.248 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T13:13:44.248 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T13:13:44.248 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T13:13:44.248 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T13:13:44.248 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T13:13:44.306 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T13:13:44.306 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T13:13:44.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-13T13:13:44.345 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-13T13:13:44.345 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-13T13:13:44.345 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-13T13:13:44.345 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-13T13:13:44.345 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-13T13:13:44.346 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:13:44.347 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T13:13:44.347 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T13:13:44.347 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T13:13:44.347 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-13T13:13:44.347 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:13:44.347 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T13:13:44.348 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T13:13:44.348 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-505.el9.x86_64 2024-09-13T13:13:44.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T13:13:44.979 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-13T13:13:45.073 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:45.112 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:45.415 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:45.415 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T13:13:45.415 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T13:13:45.415 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T13:13:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T13:13:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T13:13:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:13:46.281 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:13:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T13:13:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T13:13:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T13:13:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T13:13:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:13:46.282 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:13:46.300 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-13T13:13:46.302 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-13T13:13:46.317 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.mXQM1PZzDq 2024-09-13T13:13:46.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mXQM1PZzDq 2024-09-13T13:13:46.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.mXQM1PZzDq 2024-09-13T13:13:46.537 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.mXQM1PZzDq is 6KB 2024-09-13T13:13:46.591 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.mXQM1PZzDq 2024-09-13T13:13:46.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T13:13:46.624 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T13:13:46.667 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T13:13:46.668 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T13:13:46.668 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T13:13:46.668 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T13:13:46.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T13:13:46.705 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T13:13:46.705 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T13:13:46.705 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T13:13:46.705 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T13:13:46.705 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T13:13:46.705 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T13:13:46.705 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T13:13:46.706 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T13:13:46.765 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T13:13:46.765 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T13:13:46.765 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -qi grub2-tools 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Name : grub2-tools 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Epoch : 1 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Version : 2.06 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Release : 82.el9 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: x86_64 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Group : Unspecified 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:Size : 8274670 2024-09-13T13:13:46.801 INFO:teuthology.orchestra.run.smithi189.stdout:License : GPLv3+ 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Packager : builder@centos.org 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Vendor : CentOS 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Summary : Support tools for GRUB. 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Description : 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T13:13:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:hardware devices. 2024-09-13T13:13:46.803 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:13:46.803 INFO:teuthology.orchestra.run.smithi189.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T13:13:46.803 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T13:13:46.803 INFO:teuthology.task.kernel:Updating grub on smithi189 to boot 5.14.0-505.el9.x86_64 2024-09-13T13:13:46.804 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T13:13:46.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-13T13:13:47.498 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-09-13T13:13:48.812 INFO:teuthology.orchestra.run.smithi189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T13:13:48.845 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-09-13T13:13:48.848 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-13T13:13:48.863 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.bCjGYfTxAa 2024-09-13T13:13:48.864 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bCjGYfTxAa 2024-09-13T13:13:48.931 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.bCjGYfTxAa 2024-09-13T13:13:49.066 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.bCjGYfTxAa is 6KB 2024-09-13T13:13:49.115 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.bCjGYfTxAa 2024-09-13T13:13:49.131 DEBUG:teuthology.orchestra.run.smithi189:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T13:13:49.193 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T13:13:49.193 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T13:13:49.193 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T13:13:49.193 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T13:13:49.195 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T13:13:49.339 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-09-13T13:14:16.909 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T13:14:16.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T13:14:16.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:14:19.345 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T13:14:19.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-13T13:14:19.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:14:35.453 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-09-13T13:14:37.757 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-13T13:14:44.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T13:14:44.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:14:46.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-13T13:14:46.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:14:49.821 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-13T13:15:01.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-13T13:15:01.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:15:44.530 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T13:15:56.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T13:15:56.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:15:56.961 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-13T13:15:57.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T13:15:57.322 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T13:15:57.323 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-13T13:15:57.340 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-505.el9.x86_64 2024-09-13T13:15:57.340 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T13:15:57.340 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T13:15:57.340 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T13:15:58.341 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T13:15:58.341 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T13:15:58.414 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-09-13T13:15:58.436 DEBUG:teuthology.parallel:result is None 2024-09-13T13:16:01.846 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T13:16:16.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-13T13:16:16.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:16:17.315 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-13T13:16:17.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-13T13:16:17.702 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T13:16:17.703 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-13T13:16:17.719 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-505.el9.x86_64 2024-09-13T13:16:17.719 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T13:16:17.719 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T13:16:17.719 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T13:16:18.720 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T13:16:18.720 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T13:16:19.112 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-09-13T13:16:19.132 DEBUG:teuthology.parallel:result is None 2024-09-13T13:16:19.132 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T13:16:19.139 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T13:16:19.139 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T13:16:19.142 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T13:16:19.161 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T13:16:19.167 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T13:16:19.183 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T13:16:19.184 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T13:16:19.204 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T13:16:19.236 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T13:16:19.244 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T13:16:19.244 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T13:16:19.273 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T13:16:19.307 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T13:16:19.318 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T13:16:19.318 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T13:16:19.329 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T13:16:19.331 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T13:16:19.337 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T13:16:19.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T13:16:19.363 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T13:16:19.390 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T13:16:19.401 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T13:16:19.401 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T13:16:19.433 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T13:16:19.450 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T13:16:19.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T13:16:19.578 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T13:16:19.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T13:16:19.709 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:16:19.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T13:16:19.777 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T13:16:19.830 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T13:16:19.896 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T13:16:19.946 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T13:16:19.983 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:16:19.983 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T13:16:20.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-13T13:16:20.052 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-13T13:16:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T13:16:20.128 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T13:16:20.534 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T13:16:20.544 INFO:teuthology.task.internal:Starting timer... 2024-09-13T13:16:20.544 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T13:16:20.561 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T13:16:20.573 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-13T13:16:20.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-13T13:16:20.619 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-13T13:16:20.620 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-09-13T13:16:20.674 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-09-13T13:16:20.676 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T13:16:20.676 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-13T13:16:20.713 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-13T13:16:20.713 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-09-13T13:16:20.739 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-09-13T13:16:20.739 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-09-13T13:16:20.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T13:16:20.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:16:20.783 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T13:16:20.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:16:20.810 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T13:16:20.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-13T13:16:20.862 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-09-13T13:16:20.891 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T13:16:20.901 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T13:16:20.913 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T13:16:20.914 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T13:20:05.536 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-13T13:20:05.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T13:20:05.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:20:05.627 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-13T13:20:05.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T13:20:05.705 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-13T13:20:05.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T13:20:05.789 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-13T13:20:05.876 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-13T13:20:05.877 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T13:20:05.888 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T13:20:05.888 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T13:20:05.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T13:20:05.891 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T13:20:05.892 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T13:20:05.921 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T13:20:05.936 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T13:20:05.946 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T13:20:05.961 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T13:20:05.962 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-13T13:20:05.974 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-13T13:20:05.985 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-09-13T13:20:05.990 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T13:20:05.996 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-09-13T13:20:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T13:20:06.011 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T13:20:06.025 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T13:20:06.066 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-13T13:20:06.069 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T13:20:06.069 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-13T13:20:06.069 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.069 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.070 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.070 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.086 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-13T13:20:06.089 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T13:20:06.089 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-13T13:20:06.089 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.089 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.089 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.090 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:20:06.090 INFO:teuthology.run_tasks:Running task pexec... 2024-09-13T13:20:06.100 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-13T13:20:06.100 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T13:20:06.100 DEBUG:teuthology.orchestra.run.smithi043:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-13T13:20:06.101 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi189.front.sepia.ceph.com 2024-09-13T13:20:06.101 DEBUG:teuthology.orchestra.run.smithi189:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-13T13:20:06.478 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:20:06.549 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:Removing: 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:Removing unused dependencies: 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:Remove 2 Packages 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:Freed space: 5.5 M 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout:Removing: 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout:Removing unused dependencies: 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-13T13:20:06.551 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout:Remove 2 Packages 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout:Freed space: 5.5 M 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:20:06.552 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:20:06.555 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:20:06.556 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:20:06.565 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:20:06.565 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:20:06.694 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:20:06.703 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:20:06.774 INFO:teuthology.orchestra.run.smithi189.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-13T13:20:06.830 INFO:teuthology.orchestra.run.smithi189.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-13T13:20:06.898 INFO:teuthology.orchestra.run.smithi043.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-13T13:20:06.959 INFO:teuthology.orchestra.run.smithi043.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-13T13:20:07.385 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-13T13:20:07.385 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-13T13:20:07.479 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-13T13:20:07.480 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-13T13:20:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-13T13:20:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:07.642 INFO:teuthology.orchestra.run.smithi189.stdout:Removed: 2024-09-13T13:20:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-13T13:20:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:07.642 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:20:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-13T13:20:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:07.733 INFO:teuthology.orchestra.run.smithi043.stdout:Removed: 2024-09-13T13:20:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-13T13:20:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:07.733 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:20:08.443 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Sep 2024 01:19:20 PM UTC. 2024-09-13T13:20:08.516 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:45 ago on Fri 13 Sep 2024 01:19:23 PM UTC. 2024-09-13T13:20:08.595 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:20:08.596 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:08.596 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:20:08.596 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:08.596 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:20:08.596 INFO:teuthology.orchestra.run.smithi189.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-13T13:20:08.596 INFO:teuthology.orchestra.run.smithi189.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout:Install 7 Packages 2024-09-13T13:20:08.597 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:08.625 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 5.2 M 2024-09-13T13:20:08.626 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 19 M 2024-09-13T13:20:08.626 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:20:08.656 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:20:08.656 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-13T13:20:08.657 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-13T13:20:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:08.658 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:20:08.658 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:08.658 INFO:teuthology.orchestra.run.smithi043.stdout:Install 7 Packages 2024-09-13T13:20:08.658 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 5.2 M 2024-09-13T13:20:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 19 M 2024-09-13T13:20:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:20:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:(1/7): libnvme-1.9-3.el9.x86_64.rpm 928 kB/s | 106 kB 00:00 2024-09-13T13:20:09.178 INFO:teuthology.orchestra.run.smithi043.stdout:(2/7): nvmetcli-0.7-3.el9.noarch.rpm 301 kB/s | 44 kB 00:00 2024-09-13T13:20:09.204 INFO:teuthology.orchestra.run.smithi043.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 5.1 MB/s | 906 kB 00:00 2024-09-13T13:20:09.229 INFO:teuthology.orchestra.run.smithi043.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 856 kB/s | 72 kB 00:00 2024-09-13T13:20:09.271 INFO:teuthology.orchestra.run.smithi043.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 912 kB/s | 84 kB 00:00 2024-09-13T13:20:09.302 INFO:teuthology.orchestra.run.smithi189.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 308 kB/s | 44 kB 00:00 2024-09-13T13:20:09.327 INFO:teuthology.orchestra.run.smithi189.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 624 kB/s | 106 kB 00:00 2024-09-13T13:20:09.338 INFO:teuthology.orchestra.run.smithi043.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 6.1 MB/s | 837 kB 00:00 2024-09-13T13:20:09.353 INFO:teuthology.orchestra.run.smithi189.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.4 MB/s | 72 kB 00:00 2024-09-13T13:20:09.420 INFO:teuthology.orchestra.run.smithi189.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 915 kB/s | 84 kB 00:00 2024-09-13T13:20:09.453 INFO:teuthology.orchestra.run.smithi189.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.0 MB/s | 906 kB 00:00 2024-09-13T13:20:09.512 INFO:teuthology.orchestra.run.smithi189.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.2 MB/s | 837 kB 00:00 2024-09-13T13:20:09.580 INFO:teuthology.orchestra.run.smithi043.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 9.0 MB/s | 3.2 MB 00:00 2024-09-13T13:20:09.580 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:20:09.580 INFO:teuthology.orchestra.run.smithi043.stdout:Total 5.7 MB/s | 5.2 MB 00:00 2024-09-13T13:20:09.629 INFO:teuthology.orchestra.run.smithi189.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 15 MB/s | 3.2 MB 00:00 2024-09-13T13:20:09.629 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:20:09.629 INFO:teuthology.orchestra.run.smithi189.stdout:Total 5.2 MB/s | 5.2 MB 00:01 2024-09-13T13:20:09.699 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:20:09.715 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:20:09.715 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:20:09.741 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:20:09.756 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:20:09.756 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:20:10.018 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:20:10.018 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:20:10.071 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:20:10.071 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:20:10.533 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:20:10.544 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:20:10.609 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-13T13:20:10.631 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-13T13:20:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-13T13:20:10.666 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-13T13:20:10.679 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-13T13:20:10.694 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-13T13:20:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-13T13:20:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-13T13:20:10.942 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-13T13:20:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-13T13:20:11.391 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-13T13:20:11.420 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-13T13:20:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-13T13:20:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-13T13:20:11.537 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-13T13:20:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-13T13:20:11.597 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-13T13:20:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-13T13:20:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-13T13:20:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-13T13:20:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-13T13:20:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-13T13:20:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-13T13:20:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-13T13:20:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-13T13:20:12.655 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-13T13:20:12.656 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-13T13:20:12.656 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-13T13:20:12.656 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-13T13:20:12.656 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-13T13:20:12.656 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-13T13:20:12.656 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-13T13:20:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-13T13:20:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:12.871 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:20:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-13T13:20:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-13T13:20:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-13T13:20:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-13T13:20:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:12.872 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:20:12.997 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-13T13:20:12.997 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:12.997 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:20:12.997 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-13T13:20:12.997 INFO:teuthology.orchestra.run.smithi043.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-13T13:20:12.997 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-13T13:20:12.997 INFO:teuthology.orchestra.run.smithi043.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-13T13:20:12.998 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:12.998 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:20:13.243 DEBUG:teuthology.parallel:result is None 2024-09-13T13:20:13.359 DEBUG:teuthology.parallel:result is None 2024-09-13T13:20:13.359 INFO:teuthology.run_tasks:Running task install... 2024-09-13T13:20:13.368 DEBUG:teuthology.task.install:project ceph 2024-09-13T13:20:13.369 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0201b8c79733293453e7f10a10c7fa43119222b'}} 2024-09-13T13:20:13.369 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0201b8c79733293453e7f10a10c7fa43119222b'} 2024-09-13T13:20:13.369 INFO:teuthology.task.install:Using flavor: default 2024-09-13T13:20:13.375 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T13:20:13.375 INFO:teuthology.task.install:extra packages: [] 2024-09-13T13:20:13.375 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0201b8c79733293453e7f10a10c7fa43119222b', 'tag': None, 'wait_for_package': False} 2024-09-13T13:20:13.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:20:13.378 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0201b8c79733293453e7f10a10c7fa43119222b', 'tag': None, 'wait_for_package': False} 2024-09-13T13:20:13.378 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:20:13.533 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/ 2024-09-13T13:20:13.533 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4917.gb0201b8c 2024-09-13T13:20:13.544 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/ 2024-09-13T13:20:13.544 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4917.gb0201b8c 2024-09-13T13:20:13.651 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T13:20:13.651 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:20:13.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T13:20:13.660 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T13:20:13.660 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:20:13.661 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T13:20:13.678 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T13:20:13.679 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0201b8c79733293453e7f10a10c7fa43119222b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T13:20:13.687 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T13:20:13.687 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0201b8c79733293453e7f10a10c7fa43119222b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T13:20:13.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T13:20:13.753 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T13:20:13.830 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T13:20:13.831 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T13:20:13.857 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-13T13:20:13.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-13T13:20:13.894 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-09-13T13:20:13.895 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-09-13T13:20:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-13T13:20:14.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-09-13T13:20:14.122 INFO:teuthology.orchestra.run.smithi189.stdout:45 files removed 2024-09-13T13:20:14.149 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw 2024-09-13T13:20:14.694 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-13T13:20:14.701 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-13T13:20:14.978 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-13T13:20:14.993 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-13T13:20:15.254 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-13T13:20:15.286 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-13T13:20:16.845 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.2 MB 00:01 2024-09-13T13:20:16.977 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-13T13:20:18.730 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-13T13:20:20.228 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 20 MB 00:02 2024-09-13T13:20:23.889 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.5 MB 00:01 2024-09-13T13:20:24.554 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-13T13:20:25.531 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-13T13:20:26.080 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-13T13:20:27.018 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-13T13:20:27.409 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-13T13:20:32.824 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-13T13:20:32.856 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-13T13:20:35.236 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:20:35.238 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:20:35.238 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:35.238 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 11 M 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 3.5 M 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 3.2 M 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 5.5 M 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 22 M 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 25 k 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T13:20:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 756 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 508 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 5.5 M 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T13:20:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 45 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 143 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 162 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 321 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 305 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 99 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-13T13:20:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:35.242 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:20:35.242 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:20:35.242 INFO:teuthology.orchestra.run.smithi189.stdout:Install 31 Packages 2024-09-13T13:20:35.242 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 2 Packages 2024-09-13T13:20:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:35.242 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 62 M 2024-09-13T13:20:35.242 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:20:35.266 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 11 M 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 3.5 M 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 3.2 M 2024-09-13T13:20:35.268 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 5.5 M 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 22 M 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 25 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 756 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T13:20:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 508 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 5.5 M 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 45 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 143 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 162 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 321 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 305 k 2024-09-13T13:20:35.270 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 99 k 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout:Install 31 Packages 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-09-13T13:20:35.271 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:35.272 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 62 M 2024-09-13T13:20:35.272 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:20:35.809 INFO:teuthology.orchestra.run.smithi043.stdout:(1/33): ceph-base-19.3.0-4917.gb0201b8c.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-13T13:20:35.892 INFO:teuthology.orchestra.run.smithi043.stdout:(2/33): ceph-selinux-19.3.0-4917.gb0201b8c.el9. 302 kB/s | 25 kB 00:00 2024-09-13T13:20:36.001 INFO:teuthology.orchestra.run.smithi043.stdout:(3/33): libcephfs2-19.3.0-4917.gb0201b8c.el9.x8 6.8 MB/s | 756 kB 00:00 2024-09-13T13:20:36.126 INFO:teuthology.orchestra.run.smithi043.stdout:(4/33): ceph-radosgw-19.3.0-4917.gb0201b8c.el9. 14 MB/s | 11 MB 00:00 2024-09-13T13:20:36.159 INFO:teuthology.orchestra.run.smithi043.stdout:(5/33): libradosstriper1-19.3.0-4917.gb0201b8c. 3.1 MB/s | 508 kB 00:00 2024-09-13T13:20:36.243 INFO:teuthology.orchestra.run.smithi043.stdout:(6/33): python3-ceph-argparse-19.3.0-4917.gb020 540 kB/s | 45 kB 00:00 2024-09-13T13:20:36.343 INFO:teuthology.orchestra.run.smithi043.stdout:(7/33): python3-ceph-common-19.3.0-4917.gb0201b 1.4 MB/s | 143 kB 00:00 2024-09-13T13:20:36.460 INFO:teuthology.orchestra.run.smithi043.stdout:(8/33): librgw2-19.3.0-4917.gb0201b8c.el9.x86_6 17 MB/s | 5.5 MB 00:00 2024-09-13T13:20:36.485 INFO:teuthology.orchestra.run.smithi043.stdout:(9/33): python3-cephfs-19.3.0-4917.gb0201b8c.el 1.1 MB/s | 162 kB 00:00 2024-09-13T13:20:36.491 INFO:teuthology.orchestra.run.smithi189.stdout:(1/33): ceph-base-19.3.0-4917.gb0201b8c.el9.x86 5.0 MB/s | 5.5 MB 00:01 2024-09-13T13:20:36.519 INFO:teuthology.orchestra.run.smithi043.stdout:(10/33): python3-rados-19.3.0-4917.gb0201b8c.el 5.4 MB/s | 321 kB 00:00 2024-09-13T13:20:36.545 INFO:teuthology.orchestra.run.smithi043.stdout:(11/33): python3-rbd-19.3.0-4917.gb0201b8c.el9. 5.1 MB/s | 305 kB 00:00 2024-09-13T13:20:36.549 INFO:teuthology.orchestra.run.smithi189.stdout:(2/33): ceph-selinux-19.3.0-4917.gb0201b8c.el9. 433 kB/s | 25 kB 00:00 2024-09-13T13:20:36.578 INFO:teuthology.orchestra.run.smithi043.stdout:(12/33): python3-rgw-19.3.0-4917.gb0201b8c.el9. 1.6 MB/s | 99 kB 00:00 2024-09-13T13:20:36.658 INFO:teuthology.orchestra.run.smithi189.stdout:(3/33): libcephfs2-19.3.0-4917.gb0201b8c.el9.x8 6.8 MB/s | 756 kB 00:00 2024-09-13T13:20:36.725 INFO:teuthology.orchestra.run.smithi189.stdout:(4/33): libradosstriper1-19.3.0-4917.gb0201b8c. 7.4 MB/s | 508 kB 00:00 2024-09-13T13:20:36.833 INFO:teuthology.orchestra.run.smithi189.stdout:(5/33): ceph-radosgw-19.3.0-4917.gb0201b8c.el9. 7.6 MB/s | 11 MB 00:01 2024-09-13T13:20:36.837 INFO:teuthology.orchestra.run.smithi043.stdout:(13/33): ceph-common-19.3.0-4917.gb0201b8c.el9. 14 MB/s | 22 MB 00:01 2024-09-13T13:20:36.862 INFO:teuthology.orchestra.run.smithi043.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 337 kB/s | 107 kB 00:00 2024-09-13T13:20:36.884 INFO:teuthology.orchestra.run.smithi189.stdout:(6/33): python3-ceph-argparse-19.3.0-4917.gb020 906 kB/s | 45 kB 00:00 2024-09-13T13:20:36.912 INFO:teuthology.orchestra.run.smithi043.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 511 kB/s | 171 kB 00:00 2024-09-13T13:20:36.938 INFO:teuthology.orchestra.run.smithi043.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-13T13:20:36.942 INFO:teuthology.orchestra.run.smithi189.stdout:(7/33): python3-ceph-common-19.3.0-4917.gb0201b 2.4 MB/s | 143 kB 00:00 2024-09-13T13:20:36.971 INFO:teuthology.orchestra.run.smithi043.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-13T13:20:37.001 INFO:teuthology.orchestra.run.smithi189.stdout:(8/33): python3-cephfs-19.3.0-4917.gb0201b8c.el 2.7 MB/s | 162 kB 00:00 2024-09-13T13:20:37.005 INFO:teuthology.orchestra.run.smithi043.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-13T13:20:37.030 INFO:teuthology.orchestra.run.smithi043.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 721 kB/s | 42 kB 00:00 2024-09-13T13:20:37.055 INFO:teuthology.orchestra.run.smithi043.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-13T13:20:37.081 INFO:teuthology.orchestra.run.smithi043.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-13T13:20:37.101 INFO:teuthology.orchestra.run.smithi189.stdout:(9/33): librgw2-19.3.0-4917.gb0201b8c.el9.x86_6 15 MB/s | 5.5 MB 00:00 2024-09-13T13:20:37.106 INFO:teuthology.orchestra.run.smithi043.stdout:(22/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-13T13:20:37.126 INFO:teuthology.orchestra.run.smithi189.stdout:(10/33): python3-rados-19.3.0-4917.gb0201b8c.el 2.5 MB/s | 321 kB 00:00 2024-09-13T13:20:37.131 INFO:teuthology.orchestra.run.smithi043.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-13T13:20:37.151 INFO:teuthology.orchestra.run.smithi189.stdout:(11/33): python3-rbd-19.3.0-4917.gb0201b8c.el9. 6.0 MB/s | 305 kB 00:00 2024-09-13T13:20:37.177 INFO:teuthology.orchestra.run.smithi189.stdout:(12/33): python3-rgw-19.3.0-4917.gb0201b8c.el9. 1.9 MB/s | 99 kB 00:00 2024-09-13T13:20:37.190 INFO:teuthology.orchestra.run.smithi043.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-09-13T13:20:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-13T13:20:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:(26/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.8 MB/s | 308 kB 00:00 2024-09-13T13:20:37.282 INFO:teuthology.orchestra.run.smithi043.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-13T13:20:37.319 INFO:teuthology.orchestra.run.smithi189.stdout:(13/33): boost-program-options-1.75.0-8.el9.x86 640 kB/s | 107 kB 00:00 2024-09-13T13:20:37.332 INFO:teuthology.orchestra.run.smithi043.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T13:20:37.344 INFO:teuthology.orchestra.run.smithi189.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.0 MB/s | 171 kB 00:00 2024-09-13T13:20:37.366 INFO:teuthology.orchestra.run.smithi043.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-13T13:20:37.394 INFO:teuthology.orchestra.run.smithi189.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-09-13T13:20:37.419 INFO:teuthology.orchestra.run.smithi189.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-13T13:20:37.503 INFO:teuthology.orchestra.run.smithi189.stdout:(17/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-13T13:20:37.508 INFO:teuthology.orchestra.run.smithi043.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-09-13T13:20:37.575 INFO:teuthology.orchestra.run.smithi043.stdout:(31/33): librados2-19.3.0-4917.gb0201b8c.el9.x8 17 MB/s | 3.5 MB 00:00 2024-09-13T13:20:37.642 INFO:teuthology.orchestra.run.smithi043.stdout:(32/33): libarrow-9.0.0-11.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-09-13T13:20:37.704 INFO:teuthology.orchestra.run.smithi189.stdout:(18/33): ceph-common-19.3.0-4917.gb0201b8c.el9. 9.4 MB/s | 22 MB 00:02 2024-09-13T13:20:37.729 INFO:teuthology.orchestra.run.smithi189.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-13T13:20:37.750 INFO:teuthology.orchestra.run.smithi043.stdout:(33/33): librbd1-19.3.0-4917.gb0201b8c.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-13T13:20:37.754 INFO:teuthology.orchestra.run.smithi189.stdout:(20/33): python3-prettytable-0.7.2-27.el9.noarc 169 kB/s | 42 kB 00:00 2024-09-13T13:20:37.755 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:20:37.755 INFO:teuthology.orchestra.run.smithi043.stdout:Total 25 MB/s | 62 MB 00:02 2024-09-13T13:20:37.779 INFO:teuthology.orchestra.run.smithi189.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-13T13:20:37.813 INFO:teuthology.orchestra.run.smithi189.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-09-13T13:20:37.863 INFO:teuthology.orchestra.run.smithi189.stdout:(23/33): qatlib-service-24.02.0-1.el9.x86_64.rp 268 kB/s | 36 kB 00:00 2024-09-13T13:20:37.871 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:20:37.897 INFO:teuthology.orchestra.run.smithi189.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-13T13:20:37.912 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:20:37.912 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:20:37.922 INFO:teuthology.orchestra.run.smithi189.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-13T13:20:37.947 INFO:teuthology.orchestra.run.smithi189.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 309 kB/s | 26 kB 00:00 2024-09-13T13:20:37.973 INFO:teuthology.orchestra.run.smithi189.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-13T13:20:38.015 INFO:teuthology.orchestra.run.smithi189.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-13T13:20:38.098 INFO:teuthology.orchestra.run.smithi189.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-09-13T13:20:38.198 INFO:teuthology.orchestra.run.smithi189.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-13T13:20:38.299 INFO:teuthology.orchestra.run.smithi189.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-13T13:20:38.454 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:20:38.454 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:20:38.582 INFO:teuthology.orchestra.run.smithi189.stdout:(32/33): librados2-19.3.0-4917.gb0201b8c.el9.x8 7.2 MB/s | 3.5 MB 00:00 2024-09-13T13:20:38.707 INFO:teuthology.orchestra.run.smithi189.stdout:(33/33): librbd1-19.3.0-4917.gb0201b8c.el9.x86_ 6.4 MB/s | 3.2 MB 00:00 2024-09-13T13:20:38.714 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:20:38.714 INFO:teuthology.orchestra.run.smithi189.stdout:Total 18 MB/s | 62 MB 00:03 2024-09-13T13:20:38.833 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:20:38.871 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:20:38.871 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:20:39.401 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:20:39.401 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:20:40.024 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:20:40.102 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-13T13:20:40.130 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-13T13:20:40.155 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-13T13:20:40.180 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-13T13:20:40.200 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-13T13:20:40.374 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-13T13:20:40.426 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-13T13:20:40.787 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-13T13:20:40.890 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:20:40.891 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 7/35 2024-09-13T13:20:40.967 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 7/35 2024-09-13T13:20:40.984 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-13T13:20:41.013 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-13T13:20:41.037 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-13T13:20:41.050 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 8/35 2024-09-13T13:20:41.059 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-13T13:20:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 9/35 2024-09-13T13:20:41.085 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-13T13:20:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 9/35 2024-09-13T13:20:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-13T13:20:41.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.3.0-4917.gb0201b8c.el9. 11/35 2024-09-13T13:20:41.263 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-13T13:20:41.282 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 12/35 2024-09-13T13:20:41.311 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_6 13/35 2024-09-13T13:20:41.315 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-13T13:20:41.376 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_6 13/35 2024-09-13T13:20:41.410 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-13T13:20:41.521 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-13T13:20:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-13T13:20:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-13T13:20:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-13T13:20:41.656 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-13T13:20:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-13T13:20:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-13T13:20:41.734 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-13T13:20:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 7/35 2024-09-13T13:20:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 7/35 2024-09-13T13:20:41.946 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 8/35 2024-09-13T13:20:41.981 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 9/35 2024-09-13T13:20:42.044 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 9/35 2024-09-13T13:20:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-13T13:20:42.081 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-13T13:20:42.118 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 23/35 2024-09-13T13:20:42.125 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.3.0-4917.gb0201b8c.el9. 11/35 2024-09-13T13:20:42.196 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 12/35 2024-09-13T13:20:42.225 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_6 13/35 2024-09-13T13:20:42.227 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 23/35 2024-09-13T13:20:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 24/35 2024-09-13T13:20:42.272 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_6 13/35 2024-09-13T13:20:42.293 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-13T13:20:42.307 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-13T13:20:42.418 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-13T13:20:42.449 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-13T13:20:42.490 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-13T13:20:42.523 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-13T13:20:42.552 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-13T13:20:42.585 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-13T13:20:42.623 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-13T13:20:42.813 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-13T13:20:42.837 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 27/35 2024-09-13T13:20:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 27/35 2024-09-13T13:20:42.942 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-13T13:20:42.982 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 23/35 2024-09-13T13:20:42.989 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 28/35 2024-09-13T13:20:43.014 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.3.0-4917.gb0201b8c.el9.x8 29/35 2024-09-13T13:20:43.073 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 23/35 2024-09-13T13:20:43.103 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 24/35 2024-09-13T13:20:43.139 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-13T13:20:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-13T13:20:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 27/35 2024-09-13T13:20:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 27/35 2024-09-13T13:20:43.917 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 28/35 2024-09-13T13:20:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.3.0-4917.gb0201b8c.el9.x8 29/35 2024-09-13T13:20:45.453 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 30/35 2024-09-13T13:20:45.502 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 30/35 2024-09-13T13:20:46.102 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 30/35 2024-09-13T13:20:46.139 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 31/35 2024-09-13T13:20:46.218 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 31/35 2024-09-13T13:20:46.218 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T13:20:46.218 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T13:20:46.218 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:46.257 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 32/35 2024-09-13T13:20:46.309 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 30/35 2024-09-13T13:20:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 30/35 2024-09-13T13:20:46.927 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 30/35 2024-09-13T13:20:46.969 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 31/35 2024-09-13T13:20:47.061 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 31/35 2024-09-13T13:20:47.062 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T13:20:47.062 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T13:20:47.062 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:47.095 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 32/35 2024-09-13T13:20:56.556 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 32/35 2024-09-13T13:20:56.556 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-13T13:20:56.556 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-13T13:20:56.556 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-13T13:20:56.557 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-13T13:20:56.557 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-13T13:20:56.557 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-13T13:20:56.557 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-13T13:20:56.557 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 32/35 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-09-13T13:20:56.878 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/35 2024-09-13T13:20:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/35 2024-09-13T13:20:57.615 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:20:57.616 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T13:20:57.616 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T13:20:57.616 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T13:20:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:20:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-13T13:20:57.707 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-13T13:20:57.707 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-13T13:20:57.850 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/35 2024-09-13T13:20:57.877 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/35 2024-09-13T13:20:57.877 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:20:57.877 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T13:20:57.877 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T13:20:57.877 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T13:20:57.877 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:20:57.879 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-13T13:20:57.969 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-13T13:20:57.970 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-13T13:20:59.075 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 5/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_6 6/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 7/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4917.gb0201b8c.el9. 8/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.3.0-4917.gb0201b8c.el9.x8 9/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 10/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 11/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 12/35 2024-09-13T13:20:59.076 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 13/35 2024-09-13T13:20:59.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-13T13:20:59.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-13T13:20:59.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-13T13:20:59.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-13T13:20:59.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-13T13:20:59.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-13T13:20:59.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-13T13:20:59.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-13T13:20:59.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-13T13:20:59.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-13T13:20:59.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-13T13:20:59.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-13T13:20:59.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-13T13:20:59.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-13T13:20:59.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-13T13:20:59.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-13T13:20:59.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-13T13:20:59.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-13T13:20:59.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 32/35 2024-09-13T13:20:59.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-13T13:20:59.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 34/35 2024-09-13T13:20:59.240 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-13T13:20:59.240 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/35 2024-09-13T13:20:59.240 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 5/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_6 6/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 7/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4917.gb0201b8c.el9. 8/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.3.0-4917.gb0201b8c.el9.x8 9/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 10/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 11/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 12/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 13/35 2024-09-13T13:20:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-13T13:20:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-13T13:20:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-13T13:20:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-13T13:20:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-13T13:20:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-13T13:20:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-13T13:20:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-13T13:20:59.244 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-13T13:20:59.244 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 32/35 2024-09-13T13:20:59.244 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-13T13:20:59.244 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 34/35 2024-09-13T13:21:00.068 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T13:21:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.070 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:00.071 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-13T13:21:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:00.264 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-09-13T13:21:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.264 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T13:21:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.266 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:00.267 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:00.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-09-13T13:21:00.612 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-test 2024-09-13T13:21:01.009 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:29 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:01.147 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 50 M 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 161 k 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T13:21:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:01.149 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:01.149 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:01.149 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-13T13:21:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:01.149 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 51 M 2024-09-13T13:21:01.149 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 217 M 2024-09-13T13:21:01.150 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:01.183 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:29 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:01.328 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 50 M 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 161 k 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:01.329 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:21:01.330 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:01.330 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-09-13T13:21:01.330 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:01.330 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 51 M 2024-09-13T13:21:01.330 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 217 M 2024-09-13T13:21:01.330 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:21:01.407 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): libcephsqlite-19.3.0-4917.gb0201b8c.el9. 770 kB/s | 161 kB 00:00 2024-09-13T13:21:01.519 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-13T13:21:01.561 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-13T13:21:01.678 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): libcephsqlite-19.3.0-4917.gb0201b8c.el9. 544 kB/s | 161 kB 00:00 2024-09-13T13:21:01.783 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-13T13:21:01.916 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 423 kB/s | 304 kB 00:00 2024-09-13T13:21:02.775 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): ceph-test-19.3.0-4917.gb0201b8c.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-13T13:21:02.777 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:02.777 INFO:teuthology.orchestra.run.smithi043.stdout:Total 31 MB/s | 51 MB 00:01 2024-09-13T13:21:02.806 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:02.890 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:02.891 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:03.231 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:03.231 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:21:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:21:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T13:21:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T13:21:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/4 2024-09-13T13:21:04.411 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): ceph-test-19.3.0-4917.gb0201b8c.el9.x86_ 17 MB/s | 50 MB 00:03 2024-09-13T13:21:04.413 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:04.414 INFO:teuthology.orchestra.run.smithi189.stdout:Total 16 MB/s | 51 MB 00:03 2024-09-13T13:21:04.442 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:21:04.463 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:21:04.463 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:21:04.852 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:21:04.852 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:21:05.447 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:21:05.519 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T13:21:05.563 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T13:21:05.588 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/4 2024-09-13T13:21:09.088 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/4 2024-09-13T13:21:09.171 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/4 2024-09-13T13:21:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/4 2024-09-13T13:21:10.478 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/4 2024-09-13T13:21:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/4 2024-09-13T13:21:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/4 2024-09-13T13:21:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/4 2024-09-13T13:21:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:12.010 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:12.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-09-13T13:21:12.902 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:40 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:12.962 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/4 2024-09-13T13:21:12.962 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/4 2024-09-13T13:21:12.962 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/4 2024-09-13T13:21:12.962 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T13:21:13.044 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:13.047 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:13.047 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:21:13.047 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:13.047 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:13.047 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 6.6 k 2024-09-13T13:21:13.047 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 2.4 M 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 1.5 M 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 291 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 4.8 M 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 17 M 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T13:21:13.048 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T13:21:13.049 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T13:21:13.050 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:13.051 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:13.051 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:13.051 INFO:teuthology.orchestra.run.smithi043.stdout:Install 37 Packages 2024-09-13T13:21:13.051 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:13.052 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 31 M 2024-09-13T13:21:13.052 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 109 M 2024-09-13T13:21:13.052 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:13.239 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T13:21:13.239 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:13.239 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:13.239 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:13.239 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:13.239 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T13:21:13.240 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T13:21:13.240 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:13.240 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:13.511 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph 2024-09-13T13:21:13.636 INFO:teuthology.orchestra.run.smithi043.stdout:(1/37): ceph-19.3.0-4917.gb0201b8c.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-13T13:21:13.803 INFO:teuthology.orchestra.run.smithi043.stdout:(2/37): ceph-mgr-19.3.0-4917.gb0201b8c.el9.x86_ 5.0 MB/s | 1.5 MB 00:00 2024-09-13T13:21:13.878 INFO:teuthology.orchestra.run.smithi043.stdout:(3/37): ceph-mds-19.3.0-4917.gb0201b8c.el9.x86_ 6.3 MB/s | 2.4 MB 00:00 2024-09-13T13:21:13.962 INFO:teuthology.orchestra.run.smithi043.stdout:(4/37): ceph-mon-19.3.0-4917.gb0201b8c.el9.x86_ 15 MB/s | 4.8 MB 00:00 2024-09-13T13:21:13.987 INFO:teuthology.orchestra.run.smithi043.stdout:(5/37): ceph-mgr-modules-core-19.3.0-4917.gb020 2.6 MB/s | 291 kB 00:00 2024-09-13T13:21:14.105 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:42 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:14.248 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:14.250 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:14.250 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:21:14.250 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:14.250 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 6.6 k 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 2.4 M 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 1.5 M 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 291 k 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 4.8 M 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 17 M 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T13:21:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T13:21:14.252 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T13:21:14.253 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T13:21:14.254 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:14.254 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:21:14.254 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:14.254 INFO:teuthology.orchestra.run.smithi189.stdout:Install 37 Packages 2024-09-13T13:21:14.254 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:14.254 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 31 M 2024-09-13T13:21:14.254 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 109 M 2024-09-13T13:21:14.255 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:21:14.371 INFO:teuthology.orchestra.run.smithi043.stdout:(6/37): ceph-osd-19.3.0-4917.gb0201b8c.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-13T13:21:14.413 INFO:teuthology.orchestra.run.smithi043.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 562 kB/s | 253 kB 00:00 2024-09-13T13:21:14.538 INFO:teuthology.orchestra.run.smithi043.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-13T13:21:14.622 INFO:teuthology.orchestra.run.smithi043.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.5 MB/s | 125 kB 00:00 2024-09-13T13:21:14.647 INFO:teuthology.orchestra.run.smithi043.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 386 kB/s | 106 kB 00:00 2024-09-13T13:21:14.731 INFO:teuthology.orchestra.run.smithi043.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-09-13T13:21:14.823 INFO:teuthology.orchestra.run.smithi043.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-13T13:21:14.869 INFO:teuthology.orchestra.run.smithi189.stdout:(1/37): ceph-19.3.0-4917.gb0201b8c.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-13T13:21:14.872 INFO:teuthology.orchestra.run.smithi043.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 699 kB/s | 35 kB 00:00 2024-09-13T13:21:14.923 INFO:teuthology.orchestra.run.smithi043.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 897 kB/s | 172 kB 00:00 2024-09-13T13:21:14.948 INFO:teuthology.orchestra.run.smithi043.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-13T13:21:14.973 INFO:teuthology.orchestra.run.smithi043.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-13T13:21:14.998 INFO:teuthology.orchestra.run.smithi043.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-13T13:21:15.032 INFO:teuthology.orchestra.run.smithi043.stdout:(18/37): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:01 2024-09-13T13:21:15.070 INFO:teuthology.orchestra.run.smithi189.stdout:(2/37): ceph-mds-19.3.0-4917.gb0201b8c.el9.x86_ 7.1 MB/s | 2.4 MB 00:00 2024-09-13T13:21:15.083 INFO:teuthology.orchestra.run.smithi043.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-13T13:21:15.103 INFO:teuthology.orchestra.run.smithi189.stdout:(3/37): ceph-mgr-19.3.0-4917.gb0201b8c.el9.x86_ 4.1 MB/s | 1.5 MB 00:00 2024-09-13T13:21:15.108 INFO:teuthology.orchestra.run.smithi043.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-09-13T13:21:15.133 INFO:teuthology.orchestra.run.smithi043.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-13T13:21:15.158 INFO:teuthology.orchestra.run.smithi043.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 429 kB/s | 11 kB 00:00 2024-09-13T13:21:15.184 INFO:teuthology.orchestra.run.smithi043.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 705 kB/s | 18 kB 00:00 2024-09-13T13:21:15.203 INFO:teuthology.orchestra.run.smithi189.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4917.gb020 2.8 MB/s | 291 kB 00:00 2024-09-13T13:21:15.209 INFO:teuthology.orchestra.run.smithi043.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.5 MB/s | 358 kB 00:00 2024-09-13T13:21:15.243 INFO:teuthology.orchestra.run.smithi043.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-13T13:21:15.268 INFO:teuthology.orchestra.run.smithi043.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 379 kB/s | 19 kB 00:00 2024-09-13T13:21:15.287 INFO:teuthology.orchestra.run.smithi189.stdout:(5/37): ceph-mon-19.3.0-4917.gb0201b8c.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-13T13:21:15.293 INFO:teuthology.orchestra.run.smithi043.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-13T13:21:15.318 INFO:teuthology.orchestra.run.smithi043.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-13T13:21:15.344 INFO:teuthology.orchestra.run.smithi043.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.6 MB/s | 79 kB 00:00 2024-09-13T13:21:15.369 INFO:teuthology.orchestra.run.smithi043.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-09-13T13:21:15.394 INFO:teuthology.orchestra.run.smithi043.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T13:21:15.420 INFO:teuthology.orchestra.run.smithi043.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-13T13:21:15.437 INFO:teuthology.orchestra.run.smithi189.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-13T13:21:15.453 INFO:teuthology.orchestra.run.smithi043.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 606 kB/s | 36 kB 00:00 2024-09-13T13:21:15.478 INFO:teuthology.orchestra.run.smithi043.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-09-13T13:21:15.504 INFO:teuthology.orchestra.run.smithi043.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-13T13:21:15.529 INFO:teuthology.orchestra.run.smithi043.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.6 MB/s | 427 kB 00:00 2024-09-13T13:21:15.554 INFO:teuthology.orchestra.run.smithi043.stdout:(37/37): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-09-13T13:21:15.558 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:15.558 INFO:teuthology.orchestra.run.smithi043.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-13T13:21:15.621 INFO:teuthology.orchestra.run.smithi189.stdout:(7/37): ceph-osd-19.3.0-4917.gb0201b8c.el9.x86_ 31 MB/s | 17 MB 00:00 2024-09-13T13:21:15.646 INFO:teuthology.orchestra.run.smithi189.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 510 kB/s | 106 kB 00:00 2024-09-13T13:21:15.680 INFO:teuthology.orchestra.run.smithi189.stdout:(9/37): python3-cryptography-36.0.1-4.el9.x86_6 3.1 MB/s | 1.2 MB 00:00 2024-09-13T13:21:15.705 INFO:teuthology.orchestra.run.smithi189.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.1 MB/s | 125 kB 00:00 2024-09-13T13:21:15.714 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:15.730 INFO:teuthology.orchestra.run.smithi189.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-13T13:21:15.760 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:15.760 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:15.772 INFO:teuthology.orchestra.run.smithi189.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 895 kB/s | 135 kB 00:00 2024-09-13T13:21:16.258 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:16.258 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:21:16.331 INFO:teuthology.orchestra.run.smithi189.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 62 kB/s | 35 kB 00:00 2024-09-13T13:21:16.423 INFO:teuthology.orchestra.run.smithi189.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 263 kB/s | 188 kB 00:00 2024-09-13T13:21:16.448 INFO:teuthology.orchestra.run.smithi189.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 240 kB/s | 172 kB 00:00 2024-09-13T13:21:16.990 INFO:teuthology.orchestra.run.smithi189.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 117 kB/s | 77 kB 00:00 2024-09-13T13:21:17.024 INFO:teuthology.orchestra.run.smithi189.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2024-09-13T13:21:17.066 INFO:teuthology.orchestra.run.smithi189.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-09-13T13:21:17.108 INFO:teuthology.orchestra.run.smithi189.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-13T13:21:17.133 INFO:teuthology.orchestra.run.smithi189.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 672 kB/s | 43 kB 00:00 2024-09-13T13:21:17.158 INFO:teuthology.orchestra.run.smithi189.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-13T13:21:17.184 INFO:teuthology.orchestra.run.smithi189.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-13T13:21:17.209 INFO:teuthology.orchestra.run.smithi189.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 712 kB/s | 18 kB 00:00 2024-09-13T13:21:17.234 INFO:teuthology.orchestra.run.smithi189.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 930 kB/s | 23 kB 00:00 2024-09-13T13:21:17.259 INFO:teuthology.orchestra.run.smithi189.stdout:(25/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-13T13:21:17.284 INFO:teuthology.orchestra.run.smithi189.stdout:(26/37): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-13T13:21:17.310 INFO:teuthology.orchestra.run.smithi189.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-13T13:21:17.343 INFO:teuthology.orchestra.run.smithi189.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 235 kB/s | 20 kB 00:00 2024-09-13T13:21:17.369 INFO:teuthology.orchestra.run.smithi189.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-13T13:21:17.394 INFO:teuthology.orchestra.run.smithi189.stdout:(30/37): python3-logutils-0.3.5-21.el9.noarch.r 425 kB/s | 46 kB 00:00 2024-09-13T13:21:17.419 INFO:teuthology.orchestra.run.smithi189.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-13T13:21:17.461 INFO:teuthology.orchestra.run.smithi189.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 861 kB/s | 36 kB 00:00 2024-09-13T13:21:17.486 INFO:teuthology.orchestra.run.smithi189.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 977 kB/s | 90 kB 00:00 2024-09-13T13:21:17.511 INFO:teuthology.orchestra.run.smithi189.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-13T13:21:17.537 INFO:teuthology.orchestra.run.smithi189.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-13T13:21:17.562 INFO:teuthology.orchestra.run.smithi189.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-13T13:21:17.571 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:21:17.587 INFO:teuthology.orchestra.run.smithi189.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-13T13:21:17.593 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:17.593 INFO:teuthology.orchestra.run.smithi189.stdout:Total 9.2 MB/s | 31 MB 00:03 2024-09-13T13:21:17.640 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-13T13:21:17.673 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-13T13:21:17.709 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-13T13:21:17.746 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:21:17.784 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-13T13:21:17.786 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:21:17.786 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:21:17.825 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-13T13:21:17.860 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-13T13:21:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-13T13:21:18.040 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-13T13:21:18.094 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-13T13:21:18.137 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-13T13:21:18.169 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-13T13:21:18.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-13T13:21:18.245 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-13T13:21:18.260 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:21:18.260 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:21:18.287 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-13T13:21:18.329 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-13T13:21:18.363 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-13T13:21:18.413 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-13T13:21:18.453 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-13T13:21:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-13T13:21:18.688 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-13T13:21:18.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-13T13:21:18.831 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-13T13:21:18.869 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-13T13:21:18.927 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-13T13:21:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-13T13:21:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-13T13:21:19.248 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-13T13:21:19.328 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-13T13:21:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-13T13:21:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-13T13:21:19.560 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:21:19.605 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-13T13:21:19.623 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-13T13:21:19.664 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-13T13:21:19.692 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-13T13:21:19.763 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-13T13:21:19.776 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4917.gb0201b8c.el9. 32/37 2024-09-13T13:21:19.806 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/37 2024-09-13T13:21:19.808 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-13T13:21:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/37 2024-09-13T13:21:19.834 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:19.834 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T13:21:19.834 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T13:21:19.835 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T13:21:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:19.841 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-13T13:21:19.933 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-13T13:21:20.002 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-13T13:21:20.068 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-13T13:21:20.104 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-13T13:21:20.133 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-13T13:21:20.167 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-13T13:21:20.209 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-13T13:21:20.258 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-13T13:21:20.302 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-13T13:21:20.336 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-13T13:21:20.384 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-13T13:21:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-13T13:21:20.498 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-13T13:21:20.645 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-13T13:21:20.725 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-13T13:21:20.777 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-13T13:21:20.816 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-13T13:21:20.873 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-13T13:21:20.931 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-13T13:21:21.117 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-13T13:21:21.163 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-13T13:21:21.242 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-13T13:21:21.331 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-13T13:21:21.379 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-13T13:21:21.399 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 34/37 2024-09-13T13:21:21.428 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 34/37 2024-09-13T13:21:21.428 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:21.428 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T13:21:21.428 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T13:21:21.428 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T13:21:21.429 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:21.507 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-13T13:21:21.669 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4917.gb0201b8c.el9. 32/37 2024-09-13T13:21:21.706 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/37 2024-09-13T13:21:21.732 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 33/37 2024-09-13T13:21:21.732 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:21.732 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T13:21:21.732 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T13:21:21.733 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T13:21:21.733 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:21.908 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 35/37 2024-09-13T13:21:21.933 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 35/37 2024-09-13T13:21:21.933 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:21.934 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T13:21:21.934 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T13:21:21.934 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T13:21:21.934 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:22.175 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 36/37 2024-09-13T13:21:22.201 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 36/37 2024-09-13T13:21:22.201 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:22.201 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T13:21:22.201 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T13:21:22.201 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T13:21:22.201 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 37/37 2024-09-13T13:21:23.224 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 34/37 2024-09-13T13:21:23.252 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 34/37 2024-09-13T13:21:23.252 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:23.252 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T13:21:23.252 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T13:21:23.252 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T13:21:23.252 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:23.700 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 35/37 2024-09-13T13:21:23.724 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 35/37 2024-09-13T13:21:23.724 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:23.724 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T13:21:23.724 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T13:21:23.724 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T13:21:23.725 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:23.950 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 36/37 2024-09-13T13:21:23.974 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 36/37 2024-09-13T13:21:23.974 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:23.974 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T13:21:23.974 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T13:21:23.974 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T13:21:23.974 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:24.039 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 37/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 37/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 5/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4917.gb0201b8c.el9. 6/37 2024-09-13T13:21:24.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-13T13:21:24.046 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-13T13:21:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-13T13:21:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-13T13:21:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-13T13:21:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-13T13:21:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-13T13:21:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-13T13:21:24.047 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-13T13:21:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-13T13:21:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-13T13:21:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-13T13:21:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-13T13:21:24.048 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-13T13:21:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-13T13:21:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-13T13:21:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-13T13:21:24.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-13T13:21:25.021 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-13T13:21:25.021 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T13:21:25.022 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T13:21:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:25.024 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:25.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-09-13T13:21:25.891 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 37/37 2024-09-13T13:21:25.891 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/37 2024-09-13T13:21:25.891 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 3/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 4/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 5/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4917.gb0201b8c.el9. 6/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-13T13:21:25.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-13T13:21:25.893 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-13T13:21:25.894 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-13T13:21:25.894 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-13T13:21:25.894 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-13T13:21:25.894 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-13T13:21:25.894 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-13T13:21:25.894 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-13T13:21:25.894 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-13T13:21:25.969 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:26.063 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:21:26.109 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:26.110 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:21:26.110 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:26.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-09-13T13:21:26.766 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-13T13:21:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:26.879 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T13:21:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T13:21:26.881 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:26.882 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:26.905 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:26.905 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 784 k 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 784 k 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 791 k 2024-09-13T13:21:26.906 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:27.174 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-19.3.0-4917.gb0201b8c.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-13T13:21:27.175 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:27.175 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-13T13:21:27.175 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:27.181 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:27.181 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:27.188 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:27.188 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:21:27.266 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:21:27.282 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-base 2024-09-13T13:21:27.688 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:27.781 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:27.848 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:27.940 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-base-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:21:27.982 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:27.983 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:21:27.983 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:28.058 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install cephadm 2024-09-13T13:21:28.166 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:28.416 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:28.416 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:28.416 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:28.416 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:28.416 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:28.416 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:28.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T13:21:28.622 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:28.758 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:28.759 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:21:28.759 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 784 k 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 784 k 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 791 k 2024-09-13T13:21:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:21:29.024 INFO:teuthology.orchestra.run.smithi189.stdout:cephadm-19.3.0-4917.gb0201b8c.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-13T13:21:29.025 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:29.025 INFO:teuthology.orchestra.run.smithi189.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-13T13:21:29.025 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:21:29.030 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:21:29.030 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:21:29.036 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:21:29.037 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:21:29.109 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:21:29.190 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:57 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:29.330 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:29.331 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:29.331 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-13T13:21:29.331 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 145 k 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 145 k 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 432 k 2024-09-13T13:21:29.332 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:29.539 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-19.3.0-4917.gb0201b 705 kB/s | 145 kB 00:00 2024-09-13T13:21:29.539 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:29.539 INFO:teuthology.orchestra.run.smithi043.stdout:Total 699 kB/s | 145 kB 00:00 2024-09-13T13:21:29.540 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:29.586 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:29.609 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:29.609 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:21:29.677 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:29.756 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:21:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c. 1/1 2024-09-13T13:21:29.841 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c. 1/1 2024-09-13T13:21:29.841 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:29.841 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T13:21:29.841 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:30.029 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:30.283 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:21:30.283 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:30.284 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:30.284 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:30.284 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:30.284 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:30.523 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T13:21:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c. 1/1 2024-09-13T13:21:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:30.758 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:30.758 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:30.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-09-13T13:21:31.100 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:31.236 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:31.236 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-13T13:21:31.236 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:31.236 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:21:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 145 k 2024-09-13T13:21:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:31.237 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:21:31.237 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:31.237 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:21:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:31.237 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 145 k 2024-09-13T13:21:31.238 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 432 k 2024-09-13T13:21:31.238 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:21:31.446 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-immutable-object-cache-19.3.0-4917.gb0201b 696 kB/s | 145 kB 00:00 2024-09-13T13:21:31.446 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:31.447 INFO:teuthology.orchestra.run.smithi189.stdout:Total 692 kB/s | 145 kB 00:00 2024-09-13T13:21:31.447 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:21:31.454 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:21:31.454 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:21:31.512 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:21:31.512 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:21:31.558 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:31.651 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:21:31.661 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:21:31.696 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:31.697 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:21:31.697 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:31.730 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c. 1/1 2024-09-13T13:21:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c. 1/1 2024-09-13T13:21:31.755 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:21:31.755 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T13:21:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:31.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T13:21:32.366 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:00 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:32.505 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:32.506 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:32.506 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:21:32.506 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 14 M 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 34 k 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 16 k 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-13T13:21:32.507 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-13T13:21:32.508 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T13:21:32.508 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T13:21:32.508 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:32.508 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:32.508 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:32.508 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-13T13:21:32.508 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:32.509 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 19 M 2024-09-13T13:21:32.509 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 159 M 2024-09-13T13:21:32.509 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:32.673 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c. 1/1 2024-09-13T13:21:32.673 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:32.673 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:32.673 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:21:32.673 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:32.673 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:32.900 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr 2024-09-13T13:21:32.942 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4917.gb02 116 kB/s | 16 kB 00:00 2024-09-13T13:21:32.959 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4917.gb0 216 kB/s | 34 kB 00:00 2024-09-13T13:21:33.234 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 979 kB/s | 269 kB 00:00 2024-09-13T13:21:33.293 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-13T13:21:33.471 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:33.477 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): ceph-mgr-dashboard-19.3.0-4917.gb0201b8 21 MB/s | 14 MB 00:00 2024-09-13T13:21:33.527 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.3 MB/s | 548 kB 00:00 2024-09-13T13:21:33.569 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-mgr-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:21:33.569 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 266 kB/s | 19 kB 00:00 2024-09-13T13:21:33.594 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.3 MB/s | 863 kB 00:00 2024-09-13T13:21:33.607 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:33.608 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:21:33.608 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:33.627 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-09-13T13:21:33.653 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 526 kB/s | 31 kB 00:00 2024-09-13T13:21:33.678 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.8 MB/s | 188 kB 00:00 2024-09-13T13:21:33.682 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T13:21:33.728 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2024-09-13T13:21:33.732 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:33.732 INFO:teuthology.orchestra.run.smithi043.stdout:Total 16 MB/s | 19 MB 00:01 2024-09-13T13:21:33.782 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:33.805 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:33.805 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:34.079 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:34.079 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:21:34.245 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:02 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:34.379 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:34.380 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 14 M 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 34 k 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 16 k 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-13T13:21:34.381 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout:Install 12 Packages 2024-09-13T13:21:34.382 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:34.383 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 19 M 2024-09-13T13:21:34.383 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 159 M 2024-09-13T13:21:34.383 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:21:34.694 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:21:34.741 INFO:teuthology.orchestra.run.smithi189.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4917.gb0 225 kB/s | 34 kB 00:00 2024-09-13T13:21:34.766 INFO:teuthology.orchestra.run.smithi189.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4917.gb02 93 kB/s | 16 kB 00:00 2024-09-13T13:21:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-13T13:21:34.815 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-13T13:21:34.863 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-13T13:21:34.899 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-13T13:21:35.014 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-13T13:21:35.158 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-13T13:21:35.250 INFO:teuthology.orchestra.run.smithi189.stdout:(3/12): ceph-mgr-dashboard-19.3.0-4917.gb0201b8 21 MB/s | 14 MB 00:00 2024-09-13T13:21:35.257 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-13T13:21:35.306 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-13T13:21:35.341 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-13T13:21:35.374 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4917.gb0201b8c.el9 10/12 2024-09-13T13:21:35.650 INFO:teuthology.orchestra.run.smithi189.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 305 kB/s | 269 kB 00:00 2024-09-13T13:21:35.750 INFO:teuthology.orchestra.run.smithi189.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.4 MB/s | 548 kB 00:00 2024-09-13T13:21:35.776 INFO:teuthology.orchestra.run.smithi189.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-13T13:21:35.859 INFO:teuthology.orchestra.run.smithi189.stdout:(7/12): protobuf-3.14.0-13.el9.x86_64.rpm 924 kB/s | 1.0 MB 00:01 2024-09-13T13:21:35.901 INFO:teuthology.orchestra.run.smithi189.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 16 MB/s | 2.0 MB 00:00 2024-09-13T13:21:35.935 INFO:teuthology.orchestra.run.smithi189.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.1 MB/s | 31 kB 00:00 2024-09-13T13:21:35.968 INFO:teuthology.orchestra.run.smithi189.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.9 MB/s | 188 kB 00:00 2024-09-13T13:21:35.993 INFO:teuthology.orchestra.run.smithi189.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.1 MB/s | 144 kB 00:00 2024-09-13T13:21:36.185 INFO:teuthology.orchestra.run.smithi189.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 924 kB/s | 863 kB 00:00 2024-09-13T13:21:36.191 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:36.191 INFO:teuthology.orchestra.run.smithi189.stdout:Total 11 MB/s | 19 MB 00:01 2024-09-13T13:21:36.242 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:21:36.263 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:21:36.263 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:21:36.532 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:21:36.532 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:21:37.133 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:21:37.203 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-13T13:21:37.246 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-13T13:21:37.293 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-13T13:21:37.323 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-13T13:21:37.441 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-13T13:21:37.576 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-13T13:21:37.669 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-13T13:21:37.719 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-13T13:21:37.756 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-13T13:21:37.788 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4917.gb0201b8c.el9 10/12 2024-09-13T13:21:38.769 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4917.gb0201b8c.el 11/12 2024-09-13T13:21:38.828 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noa 12/12 2024-09-13T13:21:40.844 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noa 12/12 2024-09-13T13:21:40.844 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4917.gb0201b8c.el 1/12 2024-09-13T13:21:40.844 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noa 2/12 2024-09-13T13:21:40.845 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4917.gb0201b8c.el9 3/12 2024-09-13T13:21:40.845 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-13T13:21:40.845 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-13T13:21:40.845 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-13T13:21:40.845 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-13T13:21:40.845 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-13T13:21:40.846 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-13T13:21:40.846 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-13T13:21:40.846 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-13T13:21:41.015 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4917.gb0201b8c.el 11/12 2024-09-13T13:21:41.076 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noa 12/12 2024-09-13T13:21:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-13T13:21:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:41.165 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-13T13:21:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:41.166 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:41.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T13:21:42.005 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:10 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:42.145 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 7.4 M 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T13:21:42.146 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-13T13:21:42.147 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:42.148 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 38 M 2024-09-13T13:21:42.148 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 212 M 2024-09-13T13:21:42.148 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:42.426 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-13T13:21:42.477 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.4 MB/s | 801 kB 00:00 2024-09-13T13:21:42.552 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-13T13:21:42.602 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 366 kB/s | 18 kB 00:00 2024-09-13T13:21:42.653 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-13T13:21:42.795 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-49 15 MB/s | 7.4 MB 00:00 2024-09-13T13:21:43.028 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.4 MB/s | 3.0 MB 00:00 2024-09-13T13:21:43.053 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 947 kB/s | 245 kB 00:00 2024-09-13T13:21:43.137 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noa 12/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4917.gb0201b8c.el 1/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noa 2/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4917.gb0201b8c.el9 3/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-13T13:21:43.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-13T13:21:43.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-13T13:21:43.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-13T13:21:43.379 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 6.8 MB/s | 4.9 MB 00:00 2024-09-13T13:21:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-13T13:21:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:43.504 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-13T13:21:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:43.505 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:43.512 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-13T13:21:43.780 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T13:21:44.279 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-13T13:21:44.280 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:44.280 INFO:teuthology.orchestra.run.smithi043.stdout:Total 18 MB/s | 38 MB 00:02 2024-09-13T13:21:44.359 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:12 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:44.499 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 7.4 M 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T13:21:44.501 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout:Install 12 Packages 2024-09-13T13:21:44.502 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:44.503 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 38 M 2024-09-13T13:21:44.503 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 212 M 2024-09-13T13:21:44.503 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:21:44.646 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:44.673 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:44.673 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:45.100 INFO:teuthology.orchestra.run.smithi189.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-13T13:21:45.140 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:45.141 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:21:45.150 INFO:teuthology.orchestra.run.smithi189.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-13T13:21:45.209 INFO:teuthology.orchestra.run.smithi189.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-13T13:21:45.242 INFO:teuthology.orchestra.run.smithi189.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 548 kB/s | 18 kB 00:00 2024-09-13T13:21:45.293 INFO:teuthology.orchestra.run.smithi189.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-13T13:21:45.393 INFO:teuthology.orchestra.run.smithi189.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-49 16 MB/s | 7.4 MB 00:00 2024-09-13T13:21:45.451 INFO:teuthology.orchestra.run.smithi189.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.9 MB/s | 3.0 MB 00:00 2024-09-13T13:21:45.577 INFO:teuthology.orchestra.run.smithi189.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-09-13T13:21:45.652 INFO:teuthology.orchestra.run.smithi189.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 14 MB/s | 4.9 MB 00:00 2024-09-13T13:21:45.677 INFO:teuthology.orchestra.run.smithi189.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 862 kB/s | 245 kB 00:00 2024-09-13T13:21:45.702 INFO:teuthology.orchestra.run.smithi189.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-13T13:21:45.929 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:21:45.936 INFO:teuthology.orchestra.run.smithi189.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 57 MB/s | 16 MB 00:00 2024-09-13T13:21:45.937 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:45.937 INFO:teuthology.orchestra.run.smithi189.stdout:Total 27 MB/s | 38 MB 00:01 2024-09-13T13:21:46.013 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T13:21:46.053 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T13:21:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T13:21:46.196 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T13:21:46.288 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:21:46.314 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:21:46.314 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:21:46.390 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T13:21:46.419 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T13:21:46.506 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T13:21:46.763 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:21:46.763 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:21:47.196 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T13:21:47.321 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T13:21:47.544 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:21:47.614 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T13:21:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T13:21:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T13:21:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T13:21:47.971 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T13:21:48.001 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T13:21:48.086 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T13:21:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T13:21:48.740 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T13:21:48.856 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T13:21:49.725 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T13:21:49.751 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201 12/12 2024-09-13T13:21:50.097 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T13:21:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T13:21:51.158 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201 12/12 2024-09-13T13:21:52.265 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201 12/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201 1/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T13:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T13:21:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T13:21:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T13:21:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T13:21:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T13:21:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T13:21:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:52.691 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:53.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-09-13T13:21:53.651 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:21 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:53.765 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201 12/12 2024-09-13T13:21:53.765 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201 1/12 2024-09-13T13:21:53.765 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T13:21:53.766 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T13:21:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:53.793 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:21:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:53.793 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 49 k 2024-09-13T13:21:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:21:53.793 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-09-13T13:21:53.795 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.2 M 2024-09-13T13:21:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-09-13T13:21:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:54.035 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 276 kB/s | 26 kB 00:00 2024-09-13T13:21:54.060 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 162 kB/s | 19 kB 00:00 2024-09-13T13:21:54.119 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): ceph-mgr-rook-19.3.0-4917.gb0201b8c.el9 279 kB/s | 49 kB 00:00 2024-09-13T13:21:54.144 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-13T13:21:54.161 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T13:21:54.161 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:54.162 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:54.162 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:54.162 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T13:21:54.162 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T13:21:54.162 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T13:21:54.163 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T13:21:54.163 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T13:21:54.163 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T13:21:54.163 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T13:21:54.163 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T13:21:54.163 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T13:21:54.164 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T13:21:54.164 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T13:21:54.164 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:54.164 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:54.169 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-13T13:21:54.212 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 799 kB/s | 54 kB 00:00 2024-09-13T13:21:54.237 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 479 kB/s | 32 kB 00:00 2024-09-13T13:21:54.270 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.2 MB/s | 220 kB 00:00 2024-09-13T13:21:54.295 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-13T13:21:54.321 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-13T13:21:54.354 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2024-09-13T13:21:54.380 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 9.5 MB/s | 1.0 MB 00:00 2024-09-13T13:21:54.405 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-pyasn1-modules-0.4.8-6.el9.noa 976 kB/s | 279 kB 00:00 2024-09-13T13:21:54.406 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:54.407 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.6 MB/s | 2.2 MB 00:00 2024-09-13T13:21:54.474 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:54.490 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:54.490 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:54.574 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-rook 2024-09-13T13:21:54.589 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:54.590 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:21:55.107 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:21:55.163 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:23 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:55.278 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T13:21:55.298 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:21:55.299 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:55.299 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:21:55.299 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:55.299 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 49 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T13:21:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T13:21:55.301 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T13:21:55.301 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:55.301 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:21:55.301 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:21:55.301 INFO:teuthology.orchestra.run.smithi189.stdout:Install 13 Packages 2024-09-13T13:21:55.301 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:55.301 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 2.2 M 2024-09-13T13:21:55.302 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 26 M 2024-09-13T13:21:55.302 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:21:55.334 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T13:21:55.389 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T13:21:55.424 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T13:21:55.466 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T13:21:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T13:21:55.582 INFO:teuthology.orchestra.run.smithi189.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 162 kB/s | 19 kB 00:00 2024-09-13T13:21:55.607 INFO:teuthology.orchestra.run.smithi189.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 179 kB/s | 26 kB 00:00 2024-09-13T13:21:55.632 INFO:teuthology.orchestra.run.smithi189.stdout:(3/13): ceph-mgr-rook-19.3.0-4917.gb0201b8c.el9 292 kB/s | 49 kB 00:00 2024-09-13T13:21:55.682 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T13:21:55.724 INFO:teuthology.orchestra.run.smithi189.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-13T13:21:55.750 INFO:teuthology.orchestra.run.smithi189.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-13T13:21:55.752 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T13:21:55.774 INFO:teuthology.orchestra.run.smithi189.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-13T13:21:55.800 INFO:teuthology.orchestra.run.smithi189.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 641 kB/s | 32 kB 00:00 2024-09-13T13:21:55.825 INFO:teuthology.orchestra.run.smithi189.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 290 kB/s | 14 kB 00:00 2024-09-13T13:21:55.859 INFO:teuthology.orchestra.run.smithi189.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.8 MB/s | 220 kB 00:00 2024-09-13T13:21:55.884 INFO:teuthology.orchestra.run.smithi189.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-13T13:21:55.901 INFO:teuthology.orchestra.run.smithi189.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-13T13:21:55.951 INFO:teuthology.orchestra.run.smithi189.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.1 MB/s | 1.0 MB 00:00 2024-09-13T13:21:55.976 INFO:teuthology.orchestra.run.smithi189.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 975 kB/s | 90 kB 00:00 2024-09-13T13:21:55.979 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:55.979 INFO:teuthology.orchestra.run.smithi189.stdout:Total 3.3 MB/s | 2.2 MB 00:00 2024-09-13T13:21:56.041 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:21:56.056 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:21:56.057 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:21:56.145 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:21:56.146 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:21:56.516 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T13:21:56.573 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T13:21:56.607 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T13:21:56.655 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T13:21:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:21:56.684 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 13/13 2024-09-13T13:21:56.832 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T13:21:56.886 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T13:21:56.941 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T13:21:56.978 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T13:21:57.020 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T13:21:57.122 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T13:21:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T13:21:57.271 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T13:21:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 13/13 2024-09-13T13:21:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 1/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T13:21:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T13:21:57.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:57.773 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:21:57.996 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T13:21:58.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T13:21:58.051 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T13:21:58.085 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T13:21:58.133 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T13:21:58.155 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 13/13 2024-09-13T13:21:58.637 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:26 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:21:58.778 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:21:58.778 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 154 k 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T13:21:58.779 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:58.780 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:21:58.780 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:21:58.780 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-09-13T13:21:58.780 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:21:58.780 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 7.0 M 2024-09-13T13:21:58.780 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-09-13T13:21:58.781 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:21:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 13/13 2024-09-13T13:21:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 1/13 2024-09-13T13:21:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T13:21:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T13:21:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T13:21:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T13:21:59.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T13:21:59.327 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:59.327 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:21:59.327 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:21:59.327 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:21:59.328 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:21:59.348 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-09-13T13:21:59.432 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-13T13:21:59.457 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): ceph-mgr-cephadm-19.3.0-4917.gb0201b8c.e 613 kB/s | 154 kB 00:00 2024-09-13T13:21:59.482 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-13T13:21:59.574 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-13T13:21:59.599 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 524 kB/s | 74 kB 00:00 2024-09-13T13:21:59.602 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:21:59.602 INFO:teuthology.orchestra.run.smithi043.stdout:Total 8.5 MB/s | 7.0 MB 00:00 2024-09-13T13:21:59.606 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T13:21:59.677 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:21:59.689 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:21:59.689 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:21:59.796 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:21:59.796 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:00.114 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:00.171 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:28 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:00.297 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T13:22:00.304 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 154 k 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T13:22:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout:Install 6 Packages 2024-09-13T13:22:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:00.307 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 7.0 M 2024-09-13T13:22:00.307 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 33 M 2024-09-13T13:22:00.307 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:00.344 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T13:22:00.914 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T13:22:00.994 INFO:teuthology.orchestra.run.smithi189.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4917.gb0201b8c.e 693 kB/s | 154 kB 00:00 2024-09-13T13:22:01.006 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T13:22:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T13:22:01.111 INFO:teuthology.orchestra.run.smithi189.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.2 MB/s | 548 kB 00:00 2024-09-13T13:22:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 6/6 2024-09-13T13:22:01.144 INFO:teuthology.orchestra.run.smithi189.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.8 MB/s | 58 kB 00:00 2024-09-13T13:22:01.178 INFO:teuthology.orchestra.run.smithi189.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-13T13:22:01.578 INFO:teuthology.orchestra.run.smithi189.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 308 kB/s | 248 kB 00:00 2024-09-13T13:22:01.881 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 6/6 2024-09-13T13:22:01.881 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/6 2024-09-13T13:22:01.881 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T13:22:01.881 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T13:22:01.881 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T13:22:01.881 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T13:22:01.962 INFO:teuthology.orchestra.run.smithi189.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-09-13T13:22:01.964 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:01.964 INFO:teuthology.orchestra.run.smithi189.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-13T13:22:02.037 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:02.049 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:02.049 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:02.150 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:02.151 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T13:22:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:02.152 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:22:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T13:22:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T13:22:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T13:22:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T13:22:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T13:22:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:02.153 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:02.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-09-13T13:22:02.466 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:02.644 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T13:22:02.697 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T13:22:02.981 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:30 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:03.120 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 900 k 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:22:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:03.122 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-09-13T13:22:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:03.122 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 979 k 2024-09-13T13:22:03.122 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.9 M 2024-09-13T13:22:03.122 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:22:03.225 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T13:22:03.293 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 537 kB/s | 80 kB 00:00 2024-09-13T13:22:03.316 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T13:22:03.383 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T13:22:03.409 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 6/6 2024-09-13T13:22:03.409 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-fuse-19.3.0-4917.gb0201b8c.el9.x86_ 3.3 MB/s | 900 kB 00:00 2024-09-13T13:22:03.410 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:03.410 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.3 MB/s | 979 kB 00:00 2024-09-13T13:22:03.422 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:22:03.445 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:22:03.446 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:22:03.533 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:22:03.533 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T13:22:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/2 2024-09-13T13:22:04.222 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 6/6 2024-09-13T13:22:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 1/6 2024-09-13T13:22:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T13:22:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T13:22:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T13:22:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T13:22:04.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:04.501 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/2 2024-09-13T13:22:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/2 2024-09-13T13:22:04.800 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-fuse 2024-09-13T13:22:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T13:22:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:04.880 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T13:22:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:04.880 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:05.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-volume 2024-09-13T13:22:05.368 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:05.504 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:05.504 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:05.504 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:22:05.504 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:05.504 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:05.504 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 900 k 2024-09-13T13:22:05.504 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-13T13:22:05.505 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T13:22:05.505 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:05.505 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:05.505 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:05.505 INFO:teuthology.orchestra.run.smithi189.stdout:Install 2 Packages 2024-09-13T13:22:05.505 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:05.505 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 979 k 2024-09-13T13:22:05.506 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 2.9 M 2024-09-13T13:22:05.506 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:05.780 INFO:teuthology.orchestra.run.smithi189.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-13T13:22:05.801 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:05.913 INFO:teuthology.orchestra.run.smithi189.stdout:(2/2): ceph-fuse-19.3.0-4917.gb0201b8c.el9.x86_ 3.1 MB/s | 900 kB 00:00 2024-09-13T13:22:05.914 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:05.914 INFO:teuthology.orchestra.run.smithi189.stdout:Total 2.3 MB/s | 979 kB 00:00 2024-09-13T13:22:05.926 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:05.942 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:05.942 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 285 k 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:22:05.943 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:05.944 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 285 k 2024-09-13T13:22:05.944 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 1.3 M 2024-09-13T13:22:05.944 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:22:05.948 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:05.949 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:06.032 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:06.032 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:06.164 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-volume-19.3.0-4917.gb0201b8c.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-13T13:22:06.164 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:06.165 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-13T13:22:06.165 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:22:06.174 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:22:06.174 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:22:06.203 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:22:06.203 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:06.225 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:06.377 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T13:22:06.435 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:06.454 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/2 2024-09-13T13:22:06.516 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:22:06.532 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:22:06.532 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:22:06.532 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T13:22:06.532 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:07.176 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2/2 2024-09-13T13:22:07.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/2 2024-09-13T13:22:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T13:22:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:07.476 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T13:22:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:07.476 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:07.535 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:22:07.535 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:07.535 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:07.535 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:22:07.535 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:07.536 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:07.775 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-volume 2024-09-13T13:22:07.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-09-13T13:22:08.355 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:36 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:08.425 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:36 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:08.491 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:08.491 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.3.0-4917.gb0201b8c.el9 ceph-noarch 285 k 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:22:08.492 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:08.493 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 285 k 2024-09-13T13:22:08.493 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 1.3 M 2024-09-13T13:22:08.493 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:08.567 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-13T13:22:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:22:08.568 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 127 k 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 127 k 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 456 k 2024-09-13T13:22:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:22:08.726 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-volume-19.3.0-4917.gb0201b8c.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-13T13:22:08.727 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:08.727 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-13T13:22:08.727 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:08.735 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:08.735 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:08.769 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:08.769 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:08.793 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-19.3.0-4917.gb0201b8c.el9.x86_64 567 kB/s | 127 kB 00:00 2024-09-13T13:22:08.793 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:08.794 INFO:teuthology.orchestra.run.smithi043.stdout:Total 563 kB/s | 127 kB 00:00 2024-09-13T13:22:08.794 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:22:08.800 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:22:08.800 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:22:08.868 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:22:08.869 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:09.008 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:09.028 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:09.078 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:22:09.094 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:22:09.094 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:22:09.094 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T13:22:09.094 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:09.860 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:09.861 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:09.861 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:09.861 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:10.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-09-13T13:22:10.134 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 1/1 2024-09-13T13:22:10.134 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:10.134 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:10.134 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.3.0-4917.gb0201b8c.el9.noarch 2024-09-13T13:22:10.134 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:10.135 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:10.471 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados-devel 2024-09-13T13:22:10.715 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:10.811 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:10.857 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:10.858 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:22:10.858 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:10.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-09-13T13:22:11.060 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:39 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:11.194 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 127 k 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:22:11.195 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:11.196 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 127 k 2024-09-13T13:22:11.196 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 456 k 2024-09-13T13:22:11.196 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:11.405 INFO:teuthology.orchestra.run.smithi189.stdout:librados-devel-19.3.0-4917.gb0201b8c.el9.x86_64 606 kB/s | 127 kB 00:00 2024-09-13T13:22:11.406 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:11.406 INFO:teuthology.orchestra.run.smithi189.stdout:Total 602 kB/s | 127 kB 00:00 2024-09-13T13:22:11.406 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:11.412 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:11.412 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:11.476 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:11.476 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:11.522 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:39 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:11.662 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:11.663 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-13T13:22:11.663 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:11.663 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:22:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 32 k 2024-09-13T13:22:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:11.664 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:22:11.664 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:11.664 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:22:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:11.664 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 32 k 2024-09-13T13:22:11.664 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 142 k 2024-09-13T13:22:11.664 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:22:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:11.813 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-19.3.0-4917.gb0201b8c.el9.x86_6 212 kB/s | 32 kB 00:00 2024-09-13T13:22:11.814 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:11.814 INFO:teuthology.orchestra.run.smithi043.stdout:Total 211 kB/s | 32 kB 00:00 2024-09-13T13:22:11.814 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:22:11.818 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:22:11.818 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:22:11.835 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:22:11.836 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:12.197 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:12.446 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:12.446 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:12.447 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:12.447 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:12.447 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:12.447 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:12.676 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:12.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:12.677 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:12.677 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:12.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:12.677 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:12.725 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs2 2024-09-13T13:22:12.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-09-13T13:22:13.304 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:41 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:13.397 INFO:teuthology.orchestra.run.smithi189.stdout:Package libcephfs2-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:13.441 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:13.442 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:22:13.442 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:13.467 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:41 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:13.554 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs-devel 2024-09-13T13:22:13.562 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:13.609 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:13.610 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:22:13.610 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:13.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-09-13T13:22:14.126 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:42 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:14.262 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:14.263 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:14.263 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-13T13:22:14.263 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 32 k 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 32 k 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 142 k 2024-09-13T13:22:14.264 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:14.275 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:42 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:14.369 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:14.415 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:14.416 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:22:14.416 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:14.417 INFO:teuthology.orchestra.run.smithi189.stdout:libcephfs-devel-19.3.0-4917.gb0201b8c.el9.x86_6 207 kB/s | 32 kB 00:00 2024-09-13T13:22:14.419 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:14.419 INFO:teuthology.orchestra.run.smithi189.stdout:Total 205 kB/s | 32 kB 00:00 2024-09-13T13:22:14.419 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:14.422 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:14.422 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:14.440 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:14.440 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:14.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-09-13T13:22:14.529 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:15.045 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:15.079 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:15.220 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:15.221 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:22:15.221 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:15.281 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:15.281 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:15.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-09-13T13:22:15.473 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados2 2024-09-13T13:22:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:16.023 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:16.024 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:22:16.024 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:16.064 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:44 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:16.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-09-13T13:22:16.160 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:16.208 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:16.209 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:22:16.209 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:16.295 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librbd1 2024-09-13T13:22:16.693 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:44 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:16.787 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:16.834 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:16.835 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:22:16.835 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:16.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-09-13T13:22:16.900 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:44 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:16.995 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:17.039 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:17.040 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:22:17.040 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:17.127 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rados 2024-09-13T13:22:17.494 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:45 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:17.589 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:17.635 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:17.636 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-13T13:22:17.636 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:17.691 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:45 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:17.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-09-13T13:22:17.781 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rados-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:17.825 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:17.826 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:22:17.826 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:17.915 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rgw 2024-09-13T13:22:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:46 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:18.442 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 85 k 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:22:18.443 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:18.444 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 85 k 2024-09-13T13:22:18.444 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 231 k 2024-09-13T13:22:18.444 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:22:18.483 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:46 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:18.574 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rgw-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:18.616 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-19.3.0-4917.gb0201b8c.el9.x86_64.rpm 493 kB/s | 85 kB 00:00 2024-09-13T13:22:18.617 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:18.617 INFO:teuthology.orchestra.run.smithi043.stdout:Total 490 kB/s | 85 kB 00:00 2024-09-13T13:22:18.617 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:22:18.618 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:18.619 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:22:18.620 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:18.625 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:22:18.625 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:22:18.692 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:22:18.693 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:18.710 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-cephfs 2024-09-13T13:22:18.895 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:19.281 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:47 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:19.363 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:19.372 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-cephfs-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:19.415 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:19.416 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:22:19.416 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:19.500 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rbd 2024-09-13T13:22:19.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:19.691 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:19.691 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:19.691 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:19.691 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:19.691 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:19.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-09-13T13:22:20.069 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:48 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:20.159 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 is already installed. 2024-09-13T13:22:20.204 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:20.204 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-13T13:22:20.205 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:20.290 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-fuse 2024-09-13T13:22:20.467 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:48 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:20.606 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:20.607 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:20.607 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-13T13:22:20.607 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:20.607 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:22:20.607 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 3.2 M 2024-09-13T13:22:20.607 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:20.608 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:22:20.608 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:20.608 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:22:20.608 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:20.608 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.2 M 2024-09-13T13:22:20.608 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 14 M 2024-09-13T13:22:20.609 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:22:20.857 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:48 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:20.928 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-19.3.0-4917.gb0201b8c.el9.x86_64.rpm 10 MB/s | 3.2 MB 00:00 2024-09-13T13:22:20.929 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:20.929 INFO:teuthology.orchestra.run.smithi043.stdout:Total 9.9 MB/s | 3.2 MB 00:00 2024-09-13T13:22:20.929 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:22:20.940 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:22:20.940 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:22:20.991 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:20.992 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:20.992 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-13T13:22:20.992 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:20.992 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:20.992 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 85 k 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 85 k 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 231 k 2024-09-13T13:22:20.993 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:21.027 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:22:21.027 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:21.195 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-fuse-19.3.0-4917.gb0201b8c.el9.x86_64.rpm 420 kB/s | 85 kB 00:00 2024-09-13T13:22:21.196 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:21.196 INFO:teuthology.orchestra.run.smithi189.stdout:Total 418 kB/s | 85 kB 00:00 2024-09-13T13:22:21.197 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:21.203 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:21.203 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:21.267 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:21.268 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:21.490 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:21.506 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:21.562 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:21.590 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:21.590 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:22:21.590 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T13:22:21.590 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T13:22:21.590 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T13:22:21.590 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:21.593 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:21.980 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:22.257 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:22.257 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:22.257 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:22.257 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:22.257 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:22.258 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:22.510 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-mirror 2024-09-13T13:22:22.700 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:22.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:22.700 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:22.700 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:22.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:22.700 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:22.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-09-13T13:22:23.086 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:51 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:23.223 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:23.223 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:23.223 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-13T13:22:23.223 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:23.223 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:23.223 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 3.2 M 2024-09-13T13:22:23.224 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:23.224 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:23.224 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:23.224 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:22:23.224 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:23.224 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 3.2 M 2024-09-13T13:22:23.225 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 14 M 2024-09-13T13:22:23.225 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:23.562 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:51 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-13T13:22:23.702 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:23.702 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-13T13:22:23.702 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 170 k 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 170 k 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 490 k 2024-09-13T13:22:23.703 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-13T13:22:23.915 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-19.3.0-4917.gb0201b8c.el9.x86_64.rpm 807 kB/s | 170 kB 00:00 2024-09-13T13:22:23.915 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:23.916 INFO:teuthology.orchestra.run.smithi043.stdout:Total 800 kB/s | 170 kB 00:00 2024-09-13T13:22:23.916 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-13T13:22:23.924 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-13T13:22:23.924 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-13T13:22:23.940 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-mirror-19.3.0-4917.gb0201b8c.el9.x86_64.rpm 4.5 MB/s | 3.2 MB 00:00 2024-09-13T13:22:23.941 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:23.941 INFO:teuthology.orchestra.run.smithi189.stdout:Total 4.4 MB/s | 3.2 MB 00:00 2024-09-13T13:22:23.941 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:23.951 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:23.951 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:23.993 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-13T13:22:23.993 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-13T13:22:24.033 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:24.034 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:24.150 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-13T13:22:24.266 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:24.485 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:24.549 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:24.575 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:24.575 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T13:22:24.575 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T13:22:24.575 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T13:22:24.575 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T13:22:24.575 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:24.685 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:24.958 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:24.958 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:24.958 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-13T13:22:24.958 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:24.958 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:22:24.958 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-13T13:22:25.196 DEBUG:teuthology.parallel:result is None 2024-09-13T13:22:25.748 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:25.749 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:25.749 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:25.749 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:25.749 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:25.749 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:25.977 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-nbd 2024-09-13T13:22:26.572 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:54 ago on Fri 13 Sep 2024 01:20:32 PM UTC. 2024-09-13T13:22:26.707 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.3.0-4917.gb0201b8c.el9 ceph 170 k 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-13T13:22:26.708 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:26.709 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 170 k 2024-09-13T13:22:26.709 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 490 k 2024-09-13T13:22:26.709 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-13T13:22:26.918 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-nbd-19.3.0-4917.gb0201b8c.el9.x86_64.rpm 812 kB/s | 170 kB 00:00 2024-09-13T13:22:26.919 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:22:26.919 INFO:teuthology.orchestra.run.smithi189.stdout:Total 807 kB/s | 170 kB 00:00 2024-09-13T13:22:26.919 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-13T13:22:26.927 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-13T13:22:26.927 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-13T13:22:26.995 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-13T13:22:26.995 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-13T13:22:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-13T13:22:27.258 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:27.709 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:27.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 1/1 2024-09-13T13:22:27.992 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:27.992 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-13T13:22:27.992 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.3.0-4917.gb0201b8c.el9.x86_64 2024-09-13T13:22:27.992 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:22:27.992 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-13T13:22:28.239 DEBUG:teuthology.parallel:result is None 2024-09-13T13:22:28.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:22:28.396 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T13:22:28.422 INFO:teuthology.orchestra.run.smithi043.stdout:19.3.0-4917.gb0201b8c.el9 2024-09-13T13:22:28.423 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4917.gb0201b8c.el9 2024-09-13T13:22:28.423 INFO:teuthology.task.install:The correct ceph version 19.3.0-4917.gb0201b8c is installed. 2024-09-13T13:22:28.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:22:28.580 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T13:22:28.603 INFO:teuthology.orchestra.run.smithi189.stdout:19.3.0-4917.gb0201b8c.el9 2024-09-13T13:22:28.604 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4917.gb0201b8c.el9 2024-09-13T13:22:28.604 INFO:teuthology.task.install:The correct ceph version 19.3.0-4917.gb0201b8c is installed. 2024-09-13T13:22:28.606 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T13:22:28.606 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:22:28.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T13:22:28.638 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:22:28.638 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T13:22:28.671 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T13:22:28.671 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:22:28.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T13:22:28.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T13:22:28.777 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:22:28.777 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T13:22:28.804 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T13:22:28.870 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T13:22:28.871 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:22:28.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T13:22:28.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T13:22:28.968 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:22:28.968 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T13:22:28.994 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T13:22:29.060 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T13:22:29.060 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:22:29.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T13:22:29.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T13:22:29.157 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:22:29.157 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T13:22:29.183 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T13:22:29.247 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-13T13:22:29.342 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', 'CEPHADM_STRAY_DAEMON'], 'log-only-match': ['CEPHADM_'], 'sha1': 'b0201b8c79733293453e7f10a10c7fa43119222b', 'cephadm_mode': 'root'} 2024-09-13T13:22:29.342 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:22:29.343 INFO:tasks.cephadm:Cluster fsid is 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:22:29.343 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-13T13:22:29.343 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.c': '[v2:172.21.15.43:3301,v1:172.21.15.43:6790]', 'mon.b': '172.21.15.189'} 2024-09-13T13:22:29.343 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-09-13T13:22:29.343 INFO:tasks.cephadm:First mgr is y 2024-09-13T13:22:29.343 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-13T13:22:29.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-09-13T13:22:29.372 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-09-13T13:22:29.398 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-13T13:22:29.398 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:22:29.547 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b0201b8c79733293453e7f10a10c7fa43119222b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83042/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4917-gb0201b8c', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4917.gb0201b8c'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-13 11:13:17.876231', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-13-0840', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-13T13:22:29.695 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-guits-main-2024-09-13-0840, sha1 b0201b8c79733293453e7f10a10c7fa43119222b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:22:29.697 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/x86_64/flavors/default/cephadm 2024-09-13T13:22:29.697 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/x86_64/flavors/default/cephadm 2024-09-13T13:22:29.697 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T13:22:30.121 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 804201 Sep 13 13:22 /home/ubuntu/cephtest/cephadm 2024-09-13T13:22:30.122 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-0840/b0201b8c79733293453e7f10a10c7fa43119222b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T13:22:30.531 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 804201 Sep 13 13:22 /home/ubuntu/cephtest/cephadm 2024-09-13T13:22:30.532 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T13:22:30.552 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T13:22:30.587 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b on all hosts... 2024-09-13T13:22:30.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b pull 2024-09-13T13:22:30.595 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b pull 2024-09-13T13:22:32.197 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b... 2024-09-13T13:22:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b... 2024-09-13T13:23:01.994 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-13T13:23:01.995 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa43119222b) squid (dev)", 2024-09-13T13:23:01.995 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031", 2024-09-13T13:23:01.995 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-09-13T13:23:01.996 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b" 2024-09-13T13:23:01.996 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-13T13:23:01.996 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-13T13:23:02.287 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-13T13:23:02.287 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa43119222b) squid (dev)", 2024-09-13T13:23:02.288 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031", 2024-09-13T13:23:02.288 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-09-13T13:23:02.288 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b" 2024-09-13T13:23:02.288 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-13T13:23:02.288 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-13T13:23:02.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-09-13T13:23:02.355 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-09-13T13:23:02.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-09-13T13:23:02.421 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-09-13T13:23:02.452 INFO:tasks.cephadm:Writing seed config... 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-13T13:23:02.454 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-13T13:23:02.455 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-13T13:23:02.455 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-13T13:23:02.455 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-13T13:23:02.455 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-13T13:23:02.455 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:23:02.456 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-13T13:23:02.477 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 39cb2aa4-71d3-11ef-bceb-c7b262605968 mon election default strategy = 3 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-13T13:23:02.478 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service 2024-09-13T13:23:02.521 DEBUG:teuthology.orchestra.run.smithi043:mgr.y> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.y.service 2024-09-13T13:23:02.564 INFO:tasks.cephadm:Bootstrapping... 2024-09-13T13:23:02.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b -v bootstrap --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-13T13:23:02.775 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-13T13:23:02.775 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b', '-v', 'bootstrap', '--fsid', '39cb2aa4-71d3-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-09-13T13:23:02.776 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-13T13:23:02.776 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-09-13T13:23:02.801 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-13T13:23:02.801 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-09-13T13:23:02.801 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-09-13T13:23:02.809 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T13:23:02.809 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T13:23:02.815 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T13:23:02.816 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-13T13:23:02.822 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-13T13:23:02.829 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-13T13:23:02.829 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-13T13:23:02.829 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-09-13T13:23:02.852 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-13T13:23:02.852 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-13T13:23:02.852 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-09-13T13:23:02.852 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-09-13T13:23:02.859 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T13:23:02.859 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T13:23:02.865 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T13:23:02.865 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-13T13:23:02.872 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-13T13:23:02.879 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-13T13:23:02.879 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-13T13:23:02.879 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-09-13T13:23:02.879 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:02.879 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 140634979674768 on /run/cephadm/39cb2aa4-71d3-11ef-bceb-c7b262605968.lock 2024-09-13T13:23:02.879 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 140634979674768 acquired on /run/cephadm/39cb2aa4-71d3-11ef-bceb-c7b262605968.lock 2024-09-13T13:23:02.880 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-09-13T13:23:02.880 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-09-13T13:23:02.880 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-13T13:23:02.883 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-09-13T13:23:02.883 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-09-13T13:23:02.886 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-13T13:23:02.886 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-13T13:23:02.886 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1781sec hoplimit 64 pref medium 2024-09-13T13:23:02.888 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-13T13:23:02.888 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-13T13:23:02.889 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T13:23:02.889 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-13T13:23:02.889 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-13T13:23:02.889 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T13:23:02.889 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-13T13:23:02.890 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-13T13:23:02.890 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-13T13:23:02.890 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-13T13:23:02.890 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b... 2024-09-13T13:23:03.463 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 2024-09-13T13:23:03.463 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b... 2024-09-13T13:23:03.463 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-09-13T13:23:03.463 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-13T13:23:03.463 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:ed0c205fe07e55e926825d2d80c92003b1e841ce1f2d91029cfd1b95c3d1cf26 2024-09-13T13:23:03.463 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 2024-09-13T13:23:03.463 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-13T13:23:04.469 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa43119222b) squid (dev) 2024-09-13T13:23:04.469 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa43119222b) squid (dev) 2024-09-13T13:23:04.469 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-09-13T13:23:05.498 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-09-13T13:23:05.499 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-09-13T13:23:06.437 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQC5PORmgUQoMxAAzFIOh1dKzFqQhYF6KWxRXg== 2024-09-13T13:23:07.493 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQC6PORmwq0DMhAAhgZqE9YMt525umKKbJMB8g== 2024-09-13T13:23:08.540 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQC7PORmJa1eMxAAWpQQWcj26Sxt4kIdwrT51g== 2024-09-13T13:23:08.541 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:09.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T13:23:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:23:09.498 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-09-13T13:23:10.468 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-09-13T13:23:10.674 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-13T13:23:10.817 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-13T13:23:11.012 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968.target. 2024-09-13T13:23:11.013 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968.target. 2024-09-13T13:23:11.232 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a 2024-09-13T13:23:11.232 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-13T13:23:11.400 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968.target.wants/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@.service. 2024-09-13T13:23:11.886 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:11 smithi043 ceph-mon[29636]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T13:23:11.900 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T13:23:11.900 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-13T13:23:11.907 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T13:23:11.907 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-13T13:23:11.907 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-13T13:23:11.907 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T13:23:11.908 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-09-13T13:23:11.908 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-09-13T13:23:12.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:11 smithi043 bash[29598]: 39161aad753ffec151ce921743fee818b91faa82821c60b9554f74257ada6b03 2024-09-13T13:23:12.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:11 smithi043 systemd[1]: Started Ceph mon.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: monmap epoch 1 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: last_changed 2024-09-13T13:23:08.898338+0000 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: min_mon_release 19 (squid) 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: election_strategy: 1 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: fsmap 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T13:23:12.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: mgrmap e1: no daemons active 2024-09-13T13:23:12.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:12 smithi043 ceph-mon[29636]: from='client.? 172.21.15.43:0/2172428420' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-13T13:23:12.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.550368s) 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-13T13:23:12.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.330+0000 7f9345ec8640 1 Processor -- start 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.330+0000 7f9345ec8640 1 -- start start 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.330+0000 7f9345ec8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.330+0000 7f9345ec8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9340107000 con 0x7f9340106610 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.331+0000 7f933f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.331+0000 7f933f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49866/0 (socket says 172.21.15.43:49866) 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.331+0000 7f933f7fe640 1 -- 172.21.15.43:0/3152951286 learned_addr learned my addr 172.21.15.43:0/3152951286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:12.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.331+0000 7f933f7fe640 1 -- 172.21.15.43:0/3152951286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9340107180 con 0x7f9340106610 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f933f7fe640 1 --2- 172.21.15.43:0/3152951286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9330009820 tx=0x7f933002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148eecd0147f3add server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f933e7fc640 1 -- 172.21.15.43:0/3152951286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9330037070 con 0x7f9340106610 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f933e7fc640 1 -- 172.21.15.43:0/3152951286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330032440 con 0x7f9340106610 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f933e7fc640 1 -- 172.21.15.43:0/3152951286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9330032740 con 0x7f9340106610 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f9345ec8640 1 -- 172.21.15.43:0/3152951286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 msgr2=0x7f9340106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f9345ec8640 1 --2- 172.21.15.43:0/3152951286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9330009820 tx=0x7f933002a270 comp rx=0 tx=0).stop 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f9345ec8640 1 -- 172.21.15.43:0/3152951286 shutdown_connections 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.333+0000 7f9345ec8640 1 --2- 172.21.15.43:0/3152951286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340106a10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.334+0000 7f9345ec8640 1 -- 172.21.15.43:0/3152951286 >> 172.21.15.43:0/3152951286 conn(0x7f9340101f00 msgr2=0x7f9340104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.334+0000 7f9345ec8640 1 -- 172.21.15.43:0/3152951286 shutdown_connections 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.334+0000 7f9345ec8640 1 -- 172.21.15.43:0/3152951286 wait complete. 2024-09-13T13:23:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.334+0000 7f9345ec8640 1 Processor -- start 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.334+0000 7f9345ec8640 1 -- start start 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.334+0000 7f9345ec8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.334+0000 7f9345ec8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9340107b90 con 0x7f9340106610 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49868/0 (socket says 172.21.15.43:49868) 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933f7fe640 1 -- 172.21.15.43:0/2172428420 learned_addr learned my addr 172.21.15.43:0/2172428420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933f7fe640 1 -- 172.21.15.43:0/2172428420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934019a4c0 con 0x7f9340106610 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933f7fe640 1 --2- 172.21.15.43:0/2172428420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340199f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f93300090b0 tx=0x7f93300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933cff9640 1 -- 172.21.15.43:0/2172428420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9330041070 con 0x7f9340106610 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933cff9640 1 -- 172.21.15.43:0/2172428420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933003c7a0 con 0x7f9340106610 2024-09-13T13:23:12.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934019a750 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933cff9640 1 -- 172.21.15.43:0/2172428420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9330037040 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934019b110 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933cff9640 1 -- 172.21.15.43:0/2172428420 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9330044070 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.335+0000 7f933cff9640 1 -- 172.21.15.43:0/2172428420 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f933003d3f0 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.336+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9340108340 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.339+0000 7f933cff9640 1 -- 172.21.15.43:0/2172428420 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9330046450 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.402+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f934019b690 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.402+0000 7f933cff9640 1 -- 172.21.15.43:0/2172428420 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f9330046630 con 0x7f9340106610 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.403+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 msgr2=0x7f9340199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.403+0000 7f9345ec8640 1 --2- 172.21.15.43:0/2172428420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340199f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f93300090b0 tx=0x7f93300046c0 comp rx=0 tx=0).stop 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.403+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 shutdown_connections 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.403+0000 7f9345ec8640 1 --2- 172.21.15.43:0/2172428420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9340106610 0x7f9340199f60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:12.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.403+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 >> 172.21.15.43:0/2172428420 conn(0x7f9340101f00 msgr2=0x7f934018e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:12.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.403+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 shutdown_connections 2024-09-13T13:23:12.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:12.403+0000 7f9345ec8640 1 -- 172.21.15.43:0/2172428420 wait complete. 2024-09-13T13:23:12.989 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-09-13T13:23:12.989 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-09-13T13:23:14.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:14.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T13:23:14.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:14.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-13T13:23:14.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-13T13:23:14.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T13:23:14.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.426+0000 7fcc434e8640 1 Processor -- start 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.426+0000 7fcc434e8640 1 -- start start 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.426+0000 7fcc434e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.426+0000 7fcc434e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c107000 con 0x7fcc3c106610 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.426+0000 7fcc41284640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.426+0000 7fcc41284640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49884/0 (socket says 172.21.15.43:49884) 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.426+0000 7fcc41284640 1 -- 172.21.15.43:0/3381311735 learned_addr learned my addr 172.21.15.43:0/3381311735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.427+0000 7fcc41284640 1 -- 172.21.15.43:0/3381311735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c107180 con 0x7fcc3c106610 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.427+0000 7fcc41284640 1 --2- 172.21.15.43:0/3381311735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c106a10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009820 tx=0x7fcc3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486f63a2f0308704 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.427+0000 7fcc33fff640 1 -- 172.21.15.43:0/3381311735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc38037070 con 0x7fcc3c106610 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.427+0000 7fcc33fff640 1 -- 172.21.15.43:0/3381311735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc38032440 con 0x7fcc3c106610 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.427+0000 7fcc33fff640 1 -- 172.21.15.43:0/3381311735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc38032740 con 0x7fcc3c106610 2024-09-13T13:23:14.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.428+0000 7fcc434e8640 1 -- 172.21.15.43:0/3381311735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 msgr2=0x7fcc3c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.428+0000 7fcc434e8640 1 --2- 172.21.15.43:0/3381311735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c106a10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009820 tx=0x7fcc3802a270 comp rx=0 tx=0).stop 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.428+0000 7fcc434e8640 1 -- 172.21.15.43:0/3381311735 shutdown_connections 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.428+0000 7fcc434e8640 1 --2- 172.21.15.43:0/3381311735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c106a10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.428+0000 7fcc434e8640 1 -- 172.21.15.43:0/3381311735 >> 172.21.15.43:0/3381311735 conn(0x7fcc3c101f00 msgr2=0x7fcc3c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.428+0000 7fcc434e8640 1 -- 172.21.15.43:0/3381311735 shutdown_connections 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.428+0000 7fcc434e8640 1 -- 172.21.15.43:0/3381311735 wait complete. 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc434e8640 1 Processor -- start 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc434e8640 1 -- start start 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc434e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc434e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c107b90 con 0x7fcc3c106610 2024-09-13T13:23:14.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc41284640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc41284640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49898/0 (socket says 172.21.15.43:49898) 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc41284640 1 -- 172.21.15.43:0/1358158808 learned_addr learned my addr 172.21.15.43:0/1358158808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.429+0000 7fcc41284640 1 -- 172.21.15.43:0/1358158808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c19a4c0 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc41284640 1 --2- 172.21.15.43:0/1358158808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c199f60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcc380090b0 tx=0x7fcc380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc38041070 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3803c7a0 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c19a750 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc38037040 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c19b110 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc38044070 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.430+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc3803d3f0 con 0x7fcc3c106610 2024-09-13T13:23:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.431+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c108340 con 0x7fcc3c106610 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.433+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcc38046450 con 0x7fcc3c106610 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.495+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fcc3c19b690 con 0x7fcc3c106610 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.497+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+374 (secure 0 0 0) 0x7fcc38046630 con 0x7fcc3c106610 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.498+0000 7fcc327fc640 1 -- 172.21.15.43:0/1358158808 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fcc38058480 con 0x7fcc3c106610 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.499+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 msgr2=0x7fcc3c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.499+0000 7fcc434e8640 1 --2- 172.21.15.43:0/1358158808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c199f60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcc380090b0 tx=0x7fcc380046c0 comp rx=0 tx=0).stop 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.499+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 shutdown_connections 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.499+0000 7fcc434e8640 1 --2- 172.21.15.43:0/1358158808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc3c106610 0x7fcc3c199f60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.499+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 >> 172.21.15.43:0/1358158808 conn(0x7fcc3c101f00 msgr2=0x7fcc3c18e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:14.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.499+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 shutdown_connections 2024-09-13T13:23:14.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:13.499+0000 7fcc434e8640 1 -- 172.21.15.43:0/1358158808 wait complete. 2024-09-13T13:23:14.013 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-09-13T13:23:14.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:13 smithi043 ceph-mon[29636]: from='client.? 172.21.15.43:0/1358158808' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-13T13:23:14.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:13 smithi043 ceph-mon[29636]: from='client.? 172.21.15.43:0/1358158808' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.389+0000 7f70a6027640 1 Processor -- start 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.389+0000 7f70a6027640 1 -- start start 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.389+0000 7f70a6027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a0109270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.389+0000 7f70a6027640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70a01097b0 con 0x7f70a007a3b0 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.389+0000 7f709f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a0109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.390+0000 7f709f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a0109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49900/0 (socket says 172.21.15.43:49900) 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.390+0000 7f709f7fe640 1 -- 172.21.15.43:0/2494534635 learned_addr learned my addr 172.21.15.43:0/2494534635 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.390+0000 7f709f7fe640 1 -- 172.21.15.43:0/2494534635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a0109930 con 0x7f70a007a3b0 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.390+0000 7f709f7fe640 1 --2- 172.21.15.43:0/2494534635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a0109270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7090009e80 tx=0x7f709002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbea4173d169627 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f709e7fc640 1 -- 172.21.15.43:0/2494534635 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7090036070 con 0x7f70a007a3b0 2024-09-13T13:23:15.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f709e7fc640 1 -- 172.21.15.43:0/2494534635 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f7090031830 con 0x7f70a007a3b0 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f709e7fc640 1 -- 172.21.15.43:0/2494534635 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7090007780 con 0x7f70a007a3b0 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f70a6027640 1 -- 172.21.15.43:0/2494534635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 msgr2=0x7f70a0109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f70a6027640 1 --2- 172.21.15.43:0/2494534635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a0109270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7090009e80 tx=0x7f709002a7a0 comp rx=0 tx=0).stop 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f70a6027640 1 -- 172.21.15.43:0/2494534635 shutdown_connections 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f70a6027640 1 --2- 172.21.15.43:0/2494534635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a0109270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f70a6027640 1 -- 172.21.15.43:0/2494534635 >> 172.21.15.43:0/2494534635 conn(0x7f70a0100e00 msgr2=0x7f70a0103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f70a6027640 1 -- 172.21.15.43:0/2494534635 shutdown_connections 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.391+0000 7f70a6027640 1 -- 172.21.15.43:0/2494534635 wait complete. 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.392+0000 7f70a6027640 1 Processor -- start 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.392+0000 7f70a6027640 1 -- start start 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.392+0000 7f70a6027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.392+0000 7f70a6027640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70a010a8f0 con 0x7f70a007a3b0 2024-09-13T13:23:15.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.392+0000 7f709f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.392+0000 7f709f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49906/0 (socket says 172.21.15.43:49906) 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.392+0000 7f709f7fe640 1 -- 172.21.15.43:0/2254409841 learned_addr learned my addr 172.21.15.43:0/2254409841 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f709f7fe640 1 -- 172.21.15.43:0/2254409841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a019abb0 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f709f7fe640 1 --2- 172.21.15.43:0/2254409841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a019a650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7090009fa0 tx=0x7f70900077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f709cff9640 1 -- 172.21.15.43:0/2254409841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7090041020 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f709cff9640 1 -- 172.21.15.43:0/2254409841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f709003c470 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a019ae40 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f709cff9640 1 -- 172.21.15.43:0/2254409841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7090036040 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a019b880 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f709cff9640 1 -- 172.21.15.43:0/2254409841 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7090044020 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.393+0000 7f709cff9640 1 -- 172.21.15.43:0/2254409841 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7090046400 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.394+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a01072e0 con 0x7f70a007a3b0 2024-09-13T13:23:15.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.396+0000 7f709cff9640 1 -- 172.21.15.43:0/2254409841 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f709004b020 con 0x7f70a007a3b0 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.458+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f70a019bad0 con 0x7f70a007a3b0 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.458+0000 7f709cff9640 1 -- 172.21.15.43:0/2254409841 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f709003c900 con 0x7f70a007a3b0 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.459+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 msgr2=0x7f70a019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.459+0000 7f70a6027640 1 --2- 172.21.15.43:0/2254409841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a019a650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7090009fa0 tx=0x7f70900077a0 comp rx=0 tx=0).stop 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.459+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 shutdown_connections 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.459+0000 7f70a6027640 1 --2- 172.21.15.43:0/2254409841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a007a3b0 0x7f70a019a650 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.459+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 >> 172.21.15.43:0/2254409841 conn(0x7f70a0100e00 msgr2=0x7f70a01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.459+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 shutdown_connections 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:14.459+0000 7f70a6027640 1 -- 172.21.15.43:0/2254409841 wait complete. 2024-09-13T13:23:15.074 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-09-13T13:23:15.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:14 smithi043 ceph-mon[29636]: from='client.? 172.21.15.43:0/2254409841' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:23:15.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:15 smithi043 systemd[1]: Stopping Ceph mon.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:23:15.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a[29613]: 2024-09-13T13:23:15.342+0000 7faf3a6b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:23:15.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a[29613]: 2024-09-13T13:23:15.342+0000 7faf3a6b9640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-13T13:23:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:15 smithi043 podman[29954]: 2024-09-13 13:23:15.547889407 +0000 UTC m=+0.303814178 container died 39161aad753ffec151ce921743fee818b91faa82821c60b9554f74257ada6b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-13T13:23:16.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:15 smithi043 podman[29954]: 2024-09-13 13:23:15.873155709 +0000 UTC m=+0.629080482 container cleanup 39161aad753ffec151ce921743fee818b91faa82821c60b9554f74257ada6b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-13T13:23:16.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:15 smithi043 bash[29954]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a 2024-09-13T13:23:16.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 podman[29971]: 2024-09-13 13:23:16.057537968 +0000 UTC m=+0.508885275 container remove 39161aad753ffec151ce921743fee818b91faa82821c60b9554f74257ada6b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-13T13:23:16.622 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-13T13:23:16.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 systemd[1]: Stopped Ceph mon.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:23:16.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 systemd[1]: Starting Ceph mon.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:23:16.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 podman[30068]: 2024-09-13 13:23:16.508238447 +0000 UTC m=+0.096172733 container create 329a7351ac3999bf26927d04e6805adda411884207f7af6df9151179965f00c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-13T13:23:16.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 podman[30068]: 2024-09-13 13:23:16.42853142 +0000 UTC m=+0.016465712 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:23:16.807 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 podman[30068]: 2024-09-13 13:23:16.702295985 +0000 UTC m=+0.290230277 container init 329a7351ac3999bf26927d04e6805adda411884207f7af6df9151179965f00c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 podman[30068]: 2024-09-13 13:23:16.707093341 +0000 UTC m=+0.295027624 container start 329a7351ac3999bf26927d04e6805adda411884207f7af6df9151179965f00c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: set uid:gid to 167:167 (ceph:ceph) 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa43119222b) squid (dev), process ceph-mon, pid 7 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: pidfile_write: ignore empty --pid-file 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: load: jerasure load: lrc 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: RocksDB version: 7.9.2 2024-09-13T13:23:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Git sha 0 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Compile date 2024-09-13 10:06:52 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: DB SUMMARY 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: DB Session ID: BLQV08JRMR9JZ2D93EYZ 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: CURRENT file: CURRENT 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 91006 ; 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.error_if_exists: 0 2024-09-13T13:23:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.create_if_missing: 0 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.paranoid_checks: 1 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.env: 0x55a4953cc120 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.info_log: 0x55a49634b700 2024-09-13T13:23:16.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T13:23:16.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.statistics: (nil) 2024-09-13T13:23:16.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.use_fsync: 0 2024-09-13T13:23:16.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_log_file_size: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.allow_fallocate: 1 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.use_direct_reads: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.db_log_dir: 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.wal_dir: 2024-09-13T13:23:16.996 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.write_buffer_manager: 0x55a49634fb80 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T13:23:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.unordered_write: 0 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.row_cache: None 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.wal_filter: None 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T13:23:16.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.two_write_queues: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.wal_compression: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.atomic_flush: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.log_readahead_size: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T13:23:16.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_background_jobs: 2 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_background_compactions: -1 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_subcompactions: 1 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_open_files: -1 2024-09-13T13:23:17.000 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_background_flushes: -1 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Compression algorithms supported: 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kZSTD supported: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kXpressCompression supported: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kBZip2Compression supported: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kLZ4Compression supported: 1 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kZlibCompression supported: 1 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: kSnappyCompression supported: 1 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T13:23:17.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.merge_operator: 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_filter: None 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a49634b320) 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T13:23:17.002 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x55a49633f610 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-09-13T13:23:17.003 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-09-13T13:23:17.004 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression: NoCompression 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T13:23:17.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.num_levels: 7 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T13:23:17.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T13:23:17.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T13:23:17.008 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T13:23:17.009 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.inplace_update_support: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.bloom_locality: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.max_successive_merges: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T13:23:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.ttl: 2592000 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.enable_blob_files: false 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.min_blob_size: 0 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T13:23:17.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d9c8d6d2-c5b6-4a07-a8d4-c5e356daf22b 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726233796734423, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726233796735356, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 87471, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 269, "table_properties": {"data_size": 85619, "index_size": 236, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10710, "raw_average_key_size": 46, "raw_value_size": 79523, "raw_average_value_size": 348, "num_data_blocks": 11, "num_entries": 228, "num_filter_entries": 228, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726233796, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d9c8d6d2-c5b6-4a07-a8d4-c5e356daf22b", "db_session_id": "BLQV08JRMR9JZ2D93EYZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726233796735452, "job": 1, "event": "recovery_finished"} 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a49636ee00 2024-09-13T13:23:17.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: DB pointer 0x55a496470000 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-13T13:23:17.013 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 87.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 87.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T13:23:17.014 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 26.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 26.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x55a49633f610#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T13:23:17.015 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???) e1 preinit fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???).mds e1 new map 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???).mds e1 print_map 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout: btime 2024-09-13T13:23:11:853606+0000 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T13:23:17.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: monmap epoch 1 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: last_changed 2024-09-13T13:23:08.898338+0000 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: min_mon_release 19 (squid) 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: election_strategy: 1 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: fsmap 2024-09-13T13:23:17.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T13:23:17.018 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 ceph-mon[30106]: mgrmap e1: no daemons active 2024-09-13T13:23:17.018 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 bash[30068]: 329a7351ac3999bf26927d04e6805adda411884207f7af6df9151179965f00c8 2024-09-13T13:23:17.018 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:16 smithi043 systemd[1]: Started Ceph mon.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:23:17.861 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.257+0000 7f67a1d6e640 1 Processor -- start 2024-09-13T13:23:17.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.257+0000 7f67a1d6e640 1 -- start start 2024-09-13T13:23:17.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.257+0000 7f67a1d6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c1058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:17.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.257+0000 7f67a1d6e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f679c105eb0 con 0x7f679c1054c0 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.258+0000 7f679b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c1058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.258+0000 7f679b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c1058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41084/0 (socket says 172.21.15.43:41084) 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.258+0000 7f679b7fe640 1 -- 172.21.15.43:0/2446482093 learned_addr learned my addr 172.21.15.43:0/2446482093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.258+0000 7f679b7fe640 1 -- 172.21.15.43:0/2446482093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c106030 con 0x7f679c1054c0 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.260+0000 7f679b7fe640 1 --2- 172.21.15.43:0/2446482093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c1058c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f678c009820 tx=0x7f678c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1f980977f2a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.260+0000 7f679a7fc640 1 -- 172.21.15.43:0/2446482093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f678c037070 con 0x7f679c1054c0 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.260+0000 7f679a7fc640 1 -- 172.21.15.43:0/2446482093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f678c032870 con 0x7f679c1054c0 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.260+0000 7f679a7fc640 1 -- 172.21.15.43:0/2446482093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f678c0305d0 con 0x7f679c1054c0 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.261+0000 7f67a1d6e640 1 -- 172.21.15.43:0/2446482093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 msgr2=0x7f679c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.261+0000 7f67a1d6e640 1 --2- 172.21.15.43:0/2446482093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c1058c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f678c009820 tx=0x7f678c02a270 comp rx=0 tx=0).stop 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.261+0000 7f67a1d6e640 1 -- 172.21.15.43:0/2446482093 shutdown_connections 2024-09-13T13:23:17.866 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.261+0000 7f67a1d6e640 1 --2- 172.21.15.43:0/2446482093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c1058c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.261+0000 7f67a1d6e640 1 -- 172.21.15.43:0/2446482093 >> 172.21.15.43:0/2446482093 conn(0x7f679c100df0 msgr2=0x7f679c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.261+0000 7f67a1d6e640 1 -- 172.21.15.43:0/2446482093 shutdown_connections 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.261+0000 7f67a1d6e640 1 -- 172.21.15.43:0/2446482093 wait complete. 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f67a1d6e640 1 Processor -- start 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f67a1d6e640 1 -- start start 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f67a1d6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f67a1d6e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f679c109940 con 0x7f679c1054c0 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f679b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f679b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41088/0 (socket says 172.21.15.43:41088) 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f679b7fe640 1 -- 172.21.15.43:0/3370671062 learned_addr learned my addr 172.21.15.43:0/3370671062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.262+0000 7f679b7fe640 1 -- 172.21.15.43:0/3370671062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c19ac30 con 0x7f679c1054c0 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f679b7fe640 1 --2- 172.21.15.43:0/3370671062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c19a6d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f678c0097f0 tx=0x7f678c030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:17.867 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f6798ff9640 1 -- 172.21.15.43:0/3370671062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f678c03f020 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f6798ff9640 1 -- 172.21.15.43:0/3370671062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f678c030dc0 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679c19aec0 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f6798ff9640 1 -- 172.21.15.43:0/3370671062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f678c037040 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679c19b880 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f6798ff9640 1 -- 172.21.15.43:0/3370671062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f678c042020 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.263+0000 7f6798ff9640 1 -- 172.21.15.43:0/3370671062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f678c0448a0 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.264+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679c109ac0 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.266+0000 7f6798ff9640 1 -- 172.21.15.43:0/3370671062 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f678c049020 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.327+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f679c19c660 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.329+0000 7f6798ff9640 1 -- 172.21.15.43:0/3370671062 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f678c044400 con 0x7f679c1054c0 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.330+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 msgr2=0x7f679c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:17.868 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.330+0000 7f67a1d6e640 1 --2- 172.21.15.43:0/3370671062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c19a6d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f678c0097f0 tx=0x7f678c030b00 comp rx=0 tx=0).stop 2024-09-13T13:23:17.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.330+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 shutdown_connections 2024-09-13T13:23:17.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.330+0000 7f67a1d6e640 1 --2- 172.21.15.43:0/3370671062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679c1054c0 0x7f679c19a6d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:17.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.330+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 >> 172.21.15.43:0/3370671062 conn(0x7f679c100df0 msgr2=0x7f679c101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:17.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.330+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 shutdown_connections 2024-09-13T13:23:17.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:17.330+0000 7f67a1d6e640 1 -- 172.21.15.43:0/3370671062 wait complete. 2024-09-13T13:23:17.869 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-13T13:23:17.883 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-13T13:23:17.883 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-09-13T13:23:17.884 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-13T13:23:17.884 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-13T13:23:18.136 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.y 2024-09-13T13:23:18.136 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-13T13:23:18.285 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968.target.wants/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@.service. 2024-09-13T13:23:18.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3370671062' entity='client.admin' 2024-09-13T13:23:18.617 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:18 smithi043 podman[30343]: 2024-09-13 13:23:18.418750083 +0000 UTC m=+0.016107806 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:23:18.871 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T13:23:18.871 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-13T13:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T13:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-13T13:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-13T13:23:18.880 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T13:23:18.891 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T13:23:18.891 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-13T13:23:18.900 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T13:23:18.900 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-13T13:23:18.900 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-13T13:23:18.900 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-13T13:23:18.900 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-09-13T13:23:18.900 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-09-13T13:23:18.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:18 smithi043 podman[30343]: 2024-09-13 13:23:18.729333229 +0000 UTC m=+0.326690942 container init d9476f937c897248f0f5f157aa9a5f89c561acb501198c8387deb7577b08b8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-13T13:23:18.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:18 smithi043 podman[30343]: 2024-09-13 13:23:18.734469623 +0000 UTC m=+0.331827336 container start d9476f937c897248f0f5f157aa9a5f89c561acb501198c8387deb7577b08b8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-13T13:23:18.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:18 smithi043 bash[30343]: d9476f937c897248f0f5f157aa9a5f89c561acb501198c8387deb7577b08b8ae 2024-09-13T13:23:18.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:18 smithi043 systemd[1]: Started Ceph mgr.y for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:23:19.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2427377765' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-13T13:23:19.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "39cb2aa4-71d3-11ef-bceb-c7b262605968", 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-13T13:23:19.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:19.992 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T13:23:19.993 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T13:23:19.993 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T13:23:20.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T13:23:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-13T13:23:11:853606+0000", 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T13:23:20.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T13:23:11.854238+0000", 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.347+0000 7fe378fd4640 1 Processor -- start 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.348+0000 7fe378fd4640 1 -- start start 2024-09-13T13:23:20.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.348+0000 7fe378fd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe374109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.348+0000 7fe378fd4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741096c0 con 0x7fe374079bc0 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.348+0000 7fe37259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe374109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.348+0000 7fe37259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe374109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41116/0 (socket says 172.21.15.43:41116) 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.348+0000 7fe37259c640 1 -- 172.21.15.43:0/2647628937 learned_addr learned my addr 172.21.15.43:0/2647628937 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.349+0000 7fe37259c640 1 -- 172.21.15.43:0/2647628937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe374109840 con 0x7fe374079bc0 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.350+0000 7fe37259c640 1 --2- 172.21.15.43:0/2647628937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe374109180 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe364009820 tx=0x7fe36402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b6dbcd2558c980 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.350+0000 7fe371d9b640 1 -- 172.21.15.43:0/2647628937 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe364037070 con 0x7fe374079bc0 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.350+0000 7fe371d9b640 1 -- 172.21.15.43:0/2647628937 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe364032870 con 0x7fe374079bc0 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.350+0000 7fe371d9b640 1 -- 172.21.15.43:0/2647628937 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe364030660 con 0x7fe374079bc0 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 -- 172.21.15.43:0/2647628937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 msgr2=0x7fe374109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 --2- 172.21.15.43:0/2647628937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe374109180 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe364009820 tx=0x7fe36402a300 comp rx=0 tx=0).stop 2024-09-13T13:23:20.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 -- 172.21.15.43:0/2647628937 shutdown_connections 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 --2- 172.21.15.43:0/2647628937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe374109180 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 -- 172.21.15.43:0/2647628937 >> 172.21.15.43:0/2647628937 conn(0x7fe374100df0 msgr2=0x7fe374103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 -- 172.21.15.43:0/2647628937 shutdown_connections 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 -- 172.21.15.43:0/2647628937 wait complete. 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.351+0000 7fe378fd4640 1 Processor -- start 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe378fd4640 1 -- start start 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe378fd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe3741067d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe378fd4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410a8b0 con 0x7fe374079bc0 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe37259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe3741067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:20.005 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe37259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe3741067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41118/0 (socket says 172.21.15.43:41118) 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe37259c640 1 -- 172.21.15.43:0/2427377765 learned_addr learned my addr 172.21.15.43:0/2427377765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe37259c640 1 -- 172.21.15.43:0/2427377765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe374106d30 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe37259c640 1 --2- 172.21.15.43:0/2427377765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe3741067d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe364000c00 tx=0x7fe364030a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe36b7fe640 1 -- 172.21.15.43:0/2427377765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe364037040 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe374105120 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe36b7fe640 1 -- 172.21.15.43:0/2427377765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe364040070 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.352+0000 7fe36b7fe640 1 -- 172.21.15.43:0/2427377765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe36403a390 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.353+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3741056a0 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.353+0000 7fe36b7fe640 1 -- 172.21.15.43:0/2427377765 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe36403a610 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.353+0000 7fe36b7fe640 1 -- 172.21.15.43:0/2427377765 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe364045870 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.354+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3741071f0 con 0x7fe374079bc0 2024-09-13T13:23:20.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.356+0000 7fe36b7fe640 1 -- 172.21.15.43:0/2427377765 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe364030d50 con 0x7fe374079bc0 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.419+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe374105c10 con 0x7fe374079bc0 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.420+0000 7fe36b7fe640 1 -- 172.21.15.43:0/2427377765 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe364045ac0 con 0x7fe374079bc0 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.421+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 msgr2=0x7fe3741067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.421+0000 7fe378fd4640 1 --2- 172.21.15.43:0/2427377765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe3741067d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe364000c00 tx=0x7fe364030a60 comp rx=0 tx=0).stop 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.421+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 shutdown_connections 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.421+0000 7fe378fd4640 1 --2- 172.21.15.43:0/2427377765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe374079bc0 0x7fe3741067d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.421+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 >> 172.21.15.43:0/2427377765 conn(0x7fe374100df0 msgr2=0x7fe374101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.421+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 shutdown_connections 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:19.421+0000 7fe378fd4640 1 -- 172.21.15.43:0/2427377765 wait complete. 2024-09-13T13:23:20.007 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-09-13T13:23:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1375930427' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "39cb2aa4-71d3-11ef-bceb-c7b262605968", 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:23.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T13:23:23.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T13:23:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T13:23:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T13:23:23.040 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T13:23:23.040 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T13:23:23.040 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-13T13:23:11:853606+0000", 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T13:23:11.854238+0000", 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.435+0000 7fc4e969d640 1 Processor -- start 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.435+0000 7fc4e969d640 1 -- start start 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.435+0000 7fc4e969d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.435+0000 7fc4e969d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4109750 con 0x7fc4e407a410 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.435+0000 7fc4e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.435+0000 7fc4e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41124/0 (socket says 172.21.15.43:41124) 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.435+0000 7fc4e2ffd640 1 -- 172.21.15.43:0/316170022 learned_addr learned my addr 172.21.15.43:0/316170022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.436+0000 7fc4e2ffd640 1 -- 172.21.15.43:0/316170022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e41098d0 con 0x7fc4e407a410 2024-09-13T13:23:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.436+0000 7fc4e2ffd640 1 --2- 172.21.15.43:0/316170022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4109210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009820 tx=0x7fc4d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a550d9d428efbb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.436+0000 7fc4e27fc640 1 -- 172.21.15.43:0/316170022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d4037070 con 0x7fc4e407a410 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e27fc640 1 -- 172.21.15.43:0/316170022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fc4d4032870 con 0x7fc4e407a410 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e27fc640 1 -- 172.21.15.43:0/316170022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d40305d0 con 0x7fc4e407a410 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e969d640 1 -- 172.21.15.43:0/316170022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 msgr2=0x7fc4e4109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e969d640 1 --2- 172.21.15.43:0/316170022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4109210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009820 tx=0x7fc4d402a270 comp rx=0 tx=0).stop 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e969d640 1 -- 172.21.15.43:0/316170022 shutdown_connections 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e969d640 1 --2- 172.21.15.43:0/316170022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4109210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e969d640 1 -- 172.21.15.43:0/316170022 >> 172.21.15.43:0/316170022 conn(0x7fc4e4100e00 msgr2=0x7fc4e4103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e969d640 1 -- 172.21.15.43:0/316170022 shutdown_connections 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.437+0000 7fc4e969d640 1 -- 172.21.15.43:0/316170022 wait complete. 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.438+0000 7fc4e969d640 1 Processor -- start 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.438+0000 7fc4e969d640 1 -- start start 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.438+0000 7fc4e969d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4106860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:23.044 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.438+0000 7fc4e969d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e410a8f0 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.438+0000 7fc4e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4106860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.438+0000 7fc4e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4106860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41136/0 (socket says 172.21.15.43:41136) 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.438+0000 7fc4e2ffd640 1 -- 172.21.15.43:0/1375930427 learned_addr learned my addr 172.21.15.43:0/1375930427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.439+0000 7fc4e2ffd640 1 -- 172.21.15.43:0/1375930427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e4106dc0 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.439+0000 7fc4e2ffd640 1 --2- 172.21.15.43:0/1375930427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4106860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40090b0 tx=0x7fc4d40307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.439+0000 7fc4e0ff9640 1 -- 172.21.15.43:0/1375930427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d403f020 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.439+0000 7fc4e0ff9640 1 -- 172.21.15.43:0/1375930427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fc4d4030e20 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.439+0000 7fc4e969d640 1 -- 172.21.15.43:0/1375930427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e41051b0 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.439+0000 7fc4e0ff9640 1 -- 172.21.15.43:0/1375930427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d4037040 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.439+0000 7fc4e969d640 1 -- 172.21.15.43:0/1375930427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e4105730 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.440+0000 7fc4d9ffb640 1 -- 172.21.15.43:0/1375930427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e4107280 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.440+0000 7fc4e0ff9640 1 -- 172.21.15.43:0/1375930427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc4d4042020 con 0x7fc4e407a410 2024-09-13T13:23:23.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.440+0000 7fc4e0ff9640 1 -- 172.21.15.43:0/1375930427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4d403ab50 con 0x7fc4e407a410 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.442+0000 7fc4e0ff9640 1 -- 172.21.15.43:0/1375930427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc4d403ae40 con 0x7fc4e407a410 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.506+0000 7fc4d9ffb640 1 -- 172.21.15.43:0/1375930427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc4e4106f90 con 0x7fc4e407a410 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.506+0000 7fc4e0ff9640 1 -- 172.21.15.43:0/1375930427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fc4d403a610 con 0x7fc4e407a410 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.507+0000 7fc4d9ffb640 1 -- 172.21.15.43:0/1375930427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 msgr2=0x7fc4e4106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.507+0000 7fc4d9ffb640 1 --2- 172.21.15.43:0/1375930427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4106860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40090b0 tx=0x7fc4d40307c0 comp rx=0 tx=0).stop 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.508+0000 7fc4d9ffb640 1 -- 172.21.15.43:0/1375930427 shutdown_connections 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.508+0000 7fc4d9ffb640 1 --2- 172.21.15.43:0/1375930427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e407a410 0x7fc4e4106860 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.508+0000 7fc4d9ffb640 1 -- 172.21.15.43:0/1375930427 >> 172.21.15.43:0/1375930427 conn(0x7fc4e4100e00 msgr2=0x7fc4e41019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.508+0000 7fc4d9ffb640 1 -- 172.21.15.43:0/1375930427 shutdown_connections 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:22.508+0000 7fc4d9ffb640 1 -- 172.21.15.43:0/1375930427 wait complete. 2024-09-13T13:23:23.046 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-09-13T13:23:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: Activating manager daemon y 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: mgrmap e2: y(active, starting, since 0.00309013s) 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: Manager daemon y is now available 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' 2024-09-13T13:23:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' 2024-09-13T13:23:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:24 smithi043 ceph-mon[30106]: from='mgr.14100 172.21.15.43:0/1141762076' entity='mgr.y' 2024-09-13T13:23:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:25 smithi043 ceph-mon[30106]: mgrmap e3: y(active, since 1.00786s) 2024-09-13T13:23:26.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/970095424' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "39cb2aa4-71d3-11ef-bceb-c7b262605968", 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T13:23:26.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:26.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T13:23:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-13T13:23:11:853606+0000", 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T13:23:26.264 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T13:23:11.854238+0000", 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T13:23:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993eec1640 1 Processor -- start 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993eec1640 1 -- start start 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993eec1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f99381091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993eec1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9938109720 con 0x7f9938079b60 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993cc5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f99381091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993cc5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f99381091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41206/0 (socket says 172.21.15.43:41206) 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993cc5d640 1 -- 172.21.15.43:0/3971805848 learned_addr learned my addr 172.21.15.43:0/3971805848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.462+0000 7f993cc5d640 1 -- 172.21.15.43:0/3971805848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99381098a0 con 0x7f9938079b60 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.463+0000 7f993cc5d640 1 --2- 172.21.15.43:0/3971805848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f99381091e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9928009820 tx=0x7f992802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b052e07560d3c485 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.463+0000 7f99377fe640 1 -- 172.21.15.43:0/3971805848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9928037070 con 0x7f9938079b60 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.463+0000 7f99377fe640 1 -- 172.21.15.43:0/3971805848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f9928032870 con 0x7f9938079b60 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.463+0000 7f99377fe640 1 -- 172.21.15.43:0/3971805848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99280305d0 con 0x7f9938079b60 2024-09-13T13:23:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.464+0000 7f993eec1640 1 -- 172.21.15.43:0/3971805848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 msgr2=0x7f99381091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.464+0000 7f993eec1640 1 --2- 172.21.15.43:0/3971805848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f99381091e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9928009820 tx=0x7f992802a270 comp rx=0 tx=0).stop 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.464+0000 7f993eec1640 1 -- 172.21.15.43:0/3971805848 shutdown_connections 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.464+0000 7f993eec1640 1 --2- 172.21.15.43:0/3971805848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f99381091e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.464+0000 7f993eec1640 1 -- 172.21.15.43:0/3971805848 >> 172.21.15.43:0/3971805848 conn(0x7f9938100df0 msgr2=0x7f9938103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.464+0000 7f993eec1640 1 -- 172.21.15.43:0/3971805848 shutdown_connections 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.464+0000 7f993eec1640 1 -- 172.21.15.43:0/3971805848 wait complete. 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993eec1640 1 Processor -- start 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993eec1640 1 -- start start 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993eec1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f993819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993eec1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f993810a840 con 0x7f9938079b60 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993cc5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f993819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:26.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993cc5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f993819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41218/0 (socket says 172.21.15.43:41218) 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993cc5d640 1 -- 172.21.15.43:0/970095424 learned_addr learned my addr 172.21.15.43:0/970095424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.465+0000 7f993cc5d640 1 -- 172.21.15.43:0/970095424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993819ab80 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.466+0000 7f993cc5d640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f993819a620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f99280090b0 tx=0x7f9928030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.466+0000 7f9935ffb640 1 -- 172.21.15.43:0/970095424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f992803f020 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.466+0000 7f9935ffb640 1 -- 172.21.15.43:0/970095424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f9928030e10 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.466+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993819ae10 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.466+0000 7f9935ffb640 1 -- 172.21.15.43:0/970095424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9928037040 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.466+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993819b850 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.467+0000 7f9935ffb640 1 -- 172.21.15.43:0/970095424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 3) ==== 51648+0+0 (secure 0 0 0) 0x7f9928042020 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.467+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9938107250 con 0x7f9938079b60 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.467+0000 7f9935ffb640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f992403f4f0 0x7f99240419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:26.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.467+0000 7f9935ffb640 1 -- 172.21.15.43:0/970095424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9928074090 con 0x7f9938079b60 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.468+0000 7f9937fff640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f992403f4f0 0x7f99240419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.468+0000 7f9937fff640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f992403f4f0 0x7f99240419b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f99200098c0 tx=0x7f9920006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.472+0000 7f9935ffb640 1 -- 172.21.15.43:0/970095424 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f9928049020 con 0x7f9938079b60 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.728+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f993819c080 con 0x7f9938079b60 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.729+0000 7f9935ffb640 1 -- 172.21.15.43:0/970095424 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f992804c350 con 0x7f9938079b60 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f992403f4f0 msgr2=0x7f99240419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f992403f4f0 0x7f99240419b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f99200098c0 tx=0x7f9920006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 msgr2=0x7f993819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f993819a620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f99280090b0 tx=0x7f9928030b50 comp rx=0 tx=0).stop 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 shutdown_connections 2024-09-13T13:23:26.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f992403f4f0 0x7f99240419b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:26.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 --2- 172.21.15.43:0/970095424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938079b60 0x7f993819a620 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:26.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 >> 172.21.15.43:0/970095424 conn(0x7f9938100df0 msgr2=0x7f9938101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:26.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 shutdown_connections 2024-09-13T13:23:26.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:25.731+0000 7f993eec1640 1 -- 172.21.15.43:0/970095424 wait complete. 2024-09-13T13:23:26.270 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-09-13T13:23:27.162 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:26 smithi043 ceph-mon[30106]: mgrmap e4: y(active, since 2s) 2024-09-13T13:23:27.162 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4178810841' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T13:23:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6bd35d640 1 Processor -- start 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6bd35d640 1 -- start start 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6bd35d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b8106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6bd35d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8107000 con 0x7fb6b8106610 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b8106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b8106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43800/0 (socket says 172.21.15.43:43800) 2024-09-13T13:23:27.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6b6ffd640 1 -- 172.21.15.43:0/1954263885 learned_addr learned my addr 172.21.15.43:0/1954263885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.664+0000 7fb6b6ffd640 1 -- 172.21.15.43:0/1954263885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b8107180 con 0x7fb6b8106610 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.665+0000 7fb6b6ffd640 1 --2- 172.21.15.43:0/1954263885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b8106a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009820 tx=0x7fb6a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9419f40a1a1dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.665+0000 7fb6b67fc640 1 -- 172.21.15.43:0/1954263885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8037070 con 0x7fb6b8106610 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.665+0000 7fb6b67fc640 1 -- 172.21.15.43:0/1954263885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fb6a8032870 con 0x7fb6b8106610 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.665+0000 7fb6b67fc640 1 -- 172.21.15.43:0/1954263885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a80305d0 con 0x7fb6b8106610 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 -- 172.21.15.43:0/1954263885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 msgr2=0x7fb6b8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 --2- 172.21.15.43:0/1954263885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b8106a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009820 tx=0x7fb6a802a270 comp rx=0 tx=0).stop 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 -- 172.21.15.43:0/1954263885 shutdown_connections 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 --2- 172.21.15.43:0/1954263885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b8106a10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 -- 172.21.15.43:0/1954263885 >> 172.21.15.43:0/1954263885 conn(0x7fb6b8101f00 msgr2=0x7fb6b8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:27.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 -- 172.21.15.43:0/1954263885 shutdown_connections 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 -- 172.21.15.43:0/1954263885 wait complete. 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.666+0000 7fb6bd35d640 1 Processor -- start 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6bd35d640 1 -- start start 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6bd35d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6bd35d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b810aa90 con 0x7fb6b8106610 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43806/0 (socket says 172.21.15.43:43806) 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b6ffd640 1 -- 172.21.15.43:0/4178810841 learned_addr learned my addr 172.21.15.43:0/4178810841 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b6ffd640 1 -- 172.21.15.43:0/4178810841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b819cdb0 con 0x7fb6b8106610 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b6ffd640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b819c850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80097f0 tx=0x7fb6a8030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b4ff9640 1 -- 172.21.15.43:0/4178810841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a803f020 con 0x7fb6b8106610 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b4ff9640 1 -- 172.21.15.43:0/4178810841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fb6a8030e40 con 0x7fb6b8106610 2024-09-13T13:23:27.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b819d040 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.667+0000 7fb6b4ff9640 1 -- 172.21.15.43:0/4178810841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8037040 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.668+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b819da00 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.669+0000 7fb6b4ff9640 1 -- 172.21.15.43:0/4178810841 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 51754+0+0 (secure 0 0 0) 0x7fb6a8042020 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.669+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b810ac10 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.669+0000 7fb6b4ff9640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7fb6a403f650 0x7fb6a4041b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.669+0000 7fb6b4ff9640 1 -- 172.21.15.43:0/4178810841 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb6a80742f0 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.669+0000 7fb6affff640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7fb6a403f650 0x7fb6a4041b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.669+0000 7fb6affff640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7fb6a403f650 0x7fb6a4041b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00098c0 tx=0x7fb6a0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.674+0000 7fb6b4ff9640 1 -- 172.21.15.43:0/4178810841 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb6a8049020 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.862+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb6b819e1d0 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.863+0000 7fb6b4ff9640 1 -- 172.21.15.43:0/4178810841 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+374 (secure 0 0 0) 0x7fb6a804c350 con 0x7fb6b8106610 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.865+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7fb6a403f650 msgr2=0x7fb6a4041b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.865+0000 7fb6bd35d640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7fb6a403f650 0x7fb6a4041b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00098c0 tx=0x7fb6a0006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.865+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 msgr2=0x7fb6b819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.865+0000 7fb6bd35d640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b819c850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80097f0 tx=0x7fb6a8030b00 comp rx=0 tx=0).stop 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.866+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 shutdown_connections 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.866+0000 7fb6bd35d640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7fb6a403f650 0x7fb6a4041b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.866+0000 7fb6bd35d640 1 --2- 172.21.15.43:0/4178810841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b8106610 0x7fb6b819c850 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.866+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 >> 172.21.15.43:0/4178810841 conn(0x7fb6b8101f00 msgr2=0x7fb6b8102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.866+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 shutdown_connections 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:26.866+0000 7fb6bd35d640 1 -- 172.21.15.43:0/4178810841 wait complete. 2024-09-13T13:23:27.427 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-09-13T13:23:28.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2987260057' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-13T13:23:28.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:28 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ignoring --setuser ceph since I am not root 2024-09-13T13:23:28.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:28 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ignoring --setgroup ceph since I am not root 2024-09-13T13:23:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2987260057' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-13T13:23:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:29 smithi043 ceph-mon[30106]: mgrmap e5: y(active, since 4s) 2024-09-13T13:23:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: Active manager daemon y restarted 2024-09-13T13:23:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: Activating manager daemon y 2024-09-13T13:23:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: osdmap e2: 0 total, 0 up, 0 in 2024-09-13T13:23:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: mgrmap e6: y(active, starting, since 0.00296842s) 2024-09-13T13:23:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:23:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T13:23:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T13:23:33.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T13:23:33.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T13:23:33.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:33 smithi043 ceph-mon[30106]: Manager daemon y is now available 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.815+0000 7f836ce63640 1 Processor -- start 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.815+0000 7f836ce63640 1 -- start start 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.815+0000 7f836ce63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f8368106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.815+0000 7f836ce63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8368107000 con 0x7f8368106610 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.815+0000 7f836659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f8368106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.815+0000 7f836659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f8368106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43812/0 (socket says 172.21.15.43:43812) 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.815+0000 7f836659c640 1 -- 172.21.15.43:0/1845464554 learned_addr learned my addr 172.21.15.43:0/1845464554 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.816+0000 7f836659c640 1 -- 172.21.15.43:0/1845464554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8368107180 con 0x7f8368106610 2024-09-13T13:23:33.823 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.816+0000 7f836659c640 1 --2- 172.21.15.43:0/1845464554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f8368106a10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8358009820 tx=0x7f835802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f70f374835dcc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836559a640 1 -- 172.21.15.43:0/1845464554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8358037070 con 0x7f8368106610 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836559a640 1 -- 172.21.15.43:0/1845464554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f8358032870 con 0x7f8368106610 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836559a640 1 -- 172.21.15.43:0/1845464554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83580305d0 con 0x7f8368106610 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836ce63640 1 -- 172.21.15.43:0/1845464554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 msgr2=0x7f8368106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836ce63640 1 --2- 172.21.15.43:0/1845464554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f8368106a10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8358009820 tx=0x7f835802a270 comp rx=0 tx=0).stop 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836ce63640 1 -- 172.21.15.43:0/1845464554 shutdown_connections 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836ce63640 1 --2- 172.21.15.43:0/1845464554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f8368106a10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836ce63640 1 -- 172.21.15.43:0/1845464554 >> 172.21.15.43:0/1845464554 conn(0x7f8368101f00 msgr2=0x7f8368104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836ce63640 1 -- 172.21.15.43:0/1845464554 shutdown_connections 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.817+0000 7f836ce63640 1 -- 172.21.15.43:0/1845464554 wait complete. 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836ce63640 1 Processor -- start 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836ce63640 1 -- start start 2024-09-13T13:23:33.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836ce63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f836819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836ce63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f836810aa90 con 0x7f8368106610 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f836819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f836819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43824/0 (socket says 172.21.15.43:43824) 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836659c640 1 -- 172.21.15.43:0/2987260057 learned_addr learned my addr 172.21.15.43:0/2987260057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.818+0000 7f836659c640 1 -- 172.21.15.43:0/2987260057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836819abb0 con 0x7f8368106610 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.819+0000 7f836659c640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f836819a650 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83580097f0 tx=0x7f8358030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.819+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f835803f020 con 0x7f8368106610 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.819+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f8358030e40 con 0x7f8368106610 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.819+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8358037040 con 0x7f8368106610 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.819+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836819ae40 con 0x7f8368106610 2024-09-13T13:23:33.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.819+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836819b800 con 0x7f8368106610 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.820+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 51754+0+0 (secure 0 0 0) 0x7f8358042020 con 0x7f8368106610 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.820+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836810ac10 con 0x7f8368106610 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.820+0000 7f83577fe640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f835003f600 0x7f8350041ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.820+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8358074180 con 0x7f8368106610 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.820+0000 7f8365d9b640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f835003f600 0x7f8350041ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.820+0000 7f8365d9b640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f835003f600 0x7f8350041ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f835c0098c0 tx=0x7f835c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:27.825+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f8358049020 con 0x7f8368106610 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.062+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f836819bfd0 con 0x7f8368106610 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.114+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8358037d20 con 0x7f8368106610 2024-09-13T13:23:33.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.115+0000 7f83577fe640 1 -- 172.21.15.43:0/2987260057 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 5) ==== 51765+0+0 (secure 0 0 0) 0x7f835803ba60 con 0x7f8368106610 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f835003f600 msgr2=0x7f8350041ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f835003f600 0x7f8350041ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f835c0098c0 tx=0x7f835c006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 msgr2=0x7f836819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f836819a650 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83580097f0 tx=0x7f8358030b00 comp rx=0 tx=0).stop 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 shutdown_connections 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:6800/3041194058,v1:172.21.15.43:6801/3041194058] conn(0x7f835003f600 0x7f8350041ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 --2- 172.21.15.43:0/2987260057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8368106610 0x7f836819a650 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 >> 172.21.15.43:0/2987260057 conn(0x7f8368101f00 msgr2=0x7f8368102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 shutdown_connections 2024-09-13T13:23:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:28.116+0000 7f836ce63640 1 -- 172.21.15.43:0/2987260057 wait complete. 2024-09-13T13:23:35.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: Found migration_current of "None". Setting to last migration. 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:23:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:34 smithi043 ceph-mon[30106]: mgrmap e7: y(active, since 1.00521s) 2024-09-13T13:23:35.727 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:23:35.382+0000 7f7104c8c640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:13:23:35] ENGINE Error in HTTPServer.serve 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: Traceback (most recent call last): 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._connections.run(self.expiration_interval) 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._run(expiration_interval) 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:23:35.728 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: s = self.context.wrap_socket( 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: return self.sslsocket_class._create( 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self.do_handshake() 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._sslobj.do_handshake() 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:23:35.729 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:23:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:35 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:23:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:36 smithi043 ceph-mon[30106]: [13/Sep/2024:13:23:35] ENGINE Bus STARTING 2024-09-13T13:23:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:36 smithi043 ceph-mon[30106]: [13/Sep/2024:13:23:35] ENGINE Serving on https://172.21.15.43:7150 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:36 smithi043 ceph-mon[30106]: [13/Sep/2024:13:23:35] ENGINE Error in HTTPServer.serve 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-13T13:23:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:36 smithi043 ceph-mon[30106]: [13/Sep/2024:13:23:35] ENGINE Serving on http://172.21.15.43:8765 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:36 smithi043 ceph-mon[30106]: [13/Sep/2024:13:23:35] ENGINE Bus STARTED 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:36 smithi043 ceph-mon[30106]: mgrmap e8: y(active, since 2s) 2024-09-13T13:23:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2656096336' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-13T13:23:41.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.701+0000 7f46678f6640 1 Processor -- start 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.701+0000 7f46678f6640 1 -- start start 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.701+0000 7f46678f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f46601058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.701+0000 7f46678f6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4660105eb0 con 0x7f46601054c0 2024-09-13T13:23:41.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.701+0000 7f4665692640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f46601058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:41.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.701+0000 7f4665692640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f46601058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40700/0 (socket says 172.21.15.43:40700) 2024-09-13T13:23:41.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.701+0000 7f4665692640 1 -- 172.21.15.43:0/4245151650 learned_addr learned my addr 172.21.15.43:0/4245151650 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:41.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.702+0000 7f4665692640 1 -- 172.21.15.43:0/4245151650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4660106030 con 0x7f46601054c0 2024-09-13T13:23:41.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.702+0000 7f4665692640 1 --2- 172.21.15.43:0/4245151650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f46601058c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4658009820 tx=0x7f465802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2ec95b40911b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:41.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f465ffff640 1 -- 172.21.15.43:0/4245151650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4658037070 con 0x7f46601054c0 2024-09-13T13:23:41.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f465ffff640 1 -- 172.21.15.43:0/4245151650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f4658032870 con 0x7f46601054c0 2024-09-13T13:23:41.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f465ffff640 1 -- 172.21.15.43:0/4245151650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46580305d0 con 0x7f46601054c0 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f46678f6640 1 -- 172.21.15.43:0/4245151650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 msgr2=0x7f46601058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f46678f6640 1 --2- 172.21.15.43:0/4245151650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f46601058c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4658009820 tx=0x7f465802a270 comp rx=0 tx=0).stop 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f46678f6640 1 -- 172.21.15.43:0/4245151650 shutdown_connections 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f46678f6640 1 --2- 172.21.15.43:0/4245151650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f46601058c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f46678f6640 1 -- 172.21.15.43:0/4245151650 >> 172.21.15.43:0/4245151650 conn(0x7f4660100df0 msgr2=0x7f4660103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f46678f6640 1 -- 172.21.15.43:0/4245151650 shutdown_connections 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.703+0000 7f46678f6640 1 -- 172.21.15.43:0/4245151650 wait complete. 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.704+0000 7f46678f6640 1 Processor -- start 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.704+0000 7f46678f6640 1 -- start start 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.704+0000 7f46678f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f466019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.704+0000 7f46678f6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4660109940 con 0x7f46601054c0 2024-09-13T13:23:41.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.704+0000 7f4665692640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f466019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.704+0000 7f4665692640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f466019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40712/0 (socket says 172.21.15.43:40712) 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.704+0000 7f4665692640 1 -- 172.21.15.43:0/2656096336 learned_addr learned my addr 172.21.15.43:0/2656096336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.705+0000 7f4665692640 1 -- 172.21.15.43:0/2656096336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466019ac30 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.705+0000 7f4665692640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f466019a6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f46580090b0 tx=0x7f4658030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.705+0000 7f465e7fc640 1 -- 172.21.15.43:0/2656096336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f465803f020 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.705+0000 7f465e7fc640 1 -- 172.21.15.43:0/2656096336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f4658030e10 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.705+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466019aec0 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.705+0000 7f465e7fc640 1 -- 172.21.15.43:0/2656096336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4658037040 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.705+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466019b880 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.706+0000 7f465e7fc640 1 -- 172.21.15.43:0/2656096336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 51765+0+0 (secure 0 0 0) 0x7f4658042020 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.706+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4660109ac0 con 0x7f46601054c0 2024-09-13T13:23:41.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.706+0000 7f465e7fc640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f464803f650 0x7f4648041b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.707+0000 7f465e7fc640 1 -- 172.21.15.43:0/2656096336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4658074560 con 0x7f46601054c0 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.707+0000 7f4664e91640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f464803f650 0x7f4648041b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.707+0000 7f4664e91640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f464803f650 0x7f4648041b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46540098c0 tx=0x7f4654006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.712+0000 7f465e7fc640 1 -- 172.21.15.43:0/2656096336 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f4658049020 con 0x7f46601054c0 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.946+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f466019ba50 con 0x7f46601054c0 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.946+0000 7f465e7fc640 1 -- 172.21.15.43:0/2656096336 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v8) ==== 56+0+88 (secure 0 0 0) 0x7f4658003610 con 0x7f46601054c0 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f464803f650 msgr2=0x7f4648041b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f464803f650 0x7f4648041b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46540098c0 tx=0x7f4654006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 msgr2=0x7f466019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f466019a6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f46580090b0 tx=0x7f4658030b50 comp rx=0 tx=0).stop 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 shutdown_connections 2024-09-13T13:23:41.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f464803f650 0x7f4648041b10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:41.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 --2- 172.21.15.43:0/2656096336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46601054c0 0x7f466019a6d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:41.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.948+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 >> 172.21.15.43:0/2656096336 conn(0x7f4660100df0 msgr2=0x7f4660101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:41.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.949+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 shutdown_connections 2024-09-13T13:23:41.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:36.949+0000 7f46678f6640 1 -- 172.21.15.43:0/2656096336 wait complete. 2024-09-13T13:23:41.521 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-13T13:23:41.521 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 8... 2024-09-13T13:23:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:42 smithi043 ceph-mon[30106]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T13:23:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:42 smithi043 ceph-mon[30106]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T13:23:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-13T13:23:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f890d02d640 1 Processor -- start 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f890d02d640 1 -- start start 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f890d02d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f8908106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f890d02d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8908107000 con 0x7f8908106610 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f8906d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f8908106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f8906d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f8908106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40726/0 (socket says 172.21.15.43:40726) 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f8906d9d640 1 -- 172.21.15.43:0/451559220 learned_addr learned my addr 172.21.15.43:0/451559220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.904+0000 7f8906d9d640 1 -- 172.21.15.43:0/451559220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8908107180 con 0x7f8908106610 2024-09-13T13:23:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.905+0000 7f8906d9d640 1 --2- 172.21.15.43:0/451559220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f8908106a10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009820 tx=0x7f88f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaf14eef655cefe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.905+0000 7f8905d9b640 1 -- 172.21.15.43:0/451559220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f8037070 con 0x7f8908106610 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.905+0000 7f8905d9b640 1 -- 172.21.15.43:0/451559220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f88f8032870 con 0x7f8908106610 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.905+0000 7f8905d9b640 1 -- 172.21.15.43:0/451559220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f80305d0 con 0x7f8908106610 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.906+0000 7f890d02d640 1 -- 172.21.15.43:0/451559220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 msgr2=0x7f8908106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.906+0000 7f890d02d640 1 --2- 172.21.15.43:0/451559220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f8908106a10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009820 tx=0x7f88f802a270 comp rx=0 tx=0).stop 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.906+0000 7f890d02d640 1 -- 172.21.15.43:0/451559220 shutdown_connections 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.906+0000 7f890d02d640 1 --2- 172.21.15.43:0/451559220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f8908106a10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.906+0000 7f890d02d640 1 -- 172.21.15.43:0/451559220 >> 172.21.15.43:0/451559220 conn(0x7f8908101f00 msgr2=0x7f8908104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.906+0000 7f890d02d640 1 -- 172.21.15.43:0/451559220 shutdown_connections 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.906+0000 7f890d02d640 1 -- 172.21.15.43:0/451559220 wait complete. 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.907+0000 7f890d02d640 1 Processor -- start 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.907+0000 7f890d02d640 1 -- start start 2024-09-13T13:23:42.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.907+0000 7f890d02d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f890819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.907+0000 7f890d02d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f890810aa90 con 0x7f8908106610 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.907+0000 7f8906d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f890819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.907+0000 7f8906d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f890819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40740/0 (socket says 172.21.15.43:40740) 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.907+0000 7f8906d9d640 1 -- 172.21.15.43:0/874084360 learned_addr learned my addr 172.21.15.43:0/874084360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.908+0000 7f8906d9d640 1 -- 172.21.15.43:0/874084360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890819aba0 con 0x7f8908106610 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.908+0000 7f8906d9d640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f890819a640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f88f80097f0 tx=0x7f88f8030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.908+0000 7f88effff640 1 -- 172.21.15.43:0/874084360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f803f020 con 0x7f8908106610 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.908+0000 7f88effff640 1 -- 172.21.15.43:0/874084360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f88f8030e40 con 0x7f8908106610 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.908+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890819ae30 con 0x7f8908106610 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.908+0000 7f88effff640 1 -- 172.21.15.43:0/874084360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f8037040 con 0x7f8908106610 2024-09-13T13:23:42.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.908+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890819b7f0 con 0x7f8908106610 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.909+0000 7f88effff640 1 -- 172.21.15.43:0/874084360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 51765+0+0 (secure 0 0 0) 0x7f88f8042020 con 0x7f8908106610 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.909+0000 7f88effff640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f88f003f6a0 0x7f88f0041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.909+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8908106a90 con 0x7f88f003f6a0 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.909+0000 7f88effff640 1 -- 172.21.15.43:0/874084360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f88f8074610 con 0x7f8908106610 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.909+0000 7f890659c640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f88f003f6a0 0x7f88f0041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.910+0000 7f890659c640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f88f003f6a0 0x7f88f0041b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00ad30 tx=0x7f88fc0093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.910+0000 7f88effff640 1 -- 172.21.15.43:0/874084360 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8908106a90 con 0x7f88f003f6a0 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.917+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f890810ac10 con 0x7f88f003f6a0 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.917+0000 7f88effff640 1 -- 172.21.15.43:0/874084360 <== mgr.14118 v2:172.21.15.43:6800/85481396 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f890810ac10 con 0x7f88f003f6a0 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.917+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f88f003f6a0 msgr2=0x7f88f0041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.917+0000 7f890d02d640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f88f003f6a0 0x7f88f0041b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00ad30 tx=0x7f88fc0093f0 comp rx=0 tx=0).stop 2024-09-13T13:23:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.917+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 msgr2=0x7f890819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.917+0000 7f890d02d640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f890819a640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f88f80097f0 tx=0x7f88f8030b00 comp rx=0 tx=0).stop 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.918+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 shutdown_connections 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.918+0000 7f890d02d640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f88f003f6a0 0x7f88f0041b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.918+0000 7f890d02d640 1 --2- 172.21.15.43:0/874084360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8908106610 0x7f890819a640 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.918+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 >> 172.21.15.43:0/874084360 conn(0x7f8908101f00 msgr2=0x7f8908102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.918+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 shutdown_connections 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:41.918+0000 7f890d02d640 1 -- 172.21.15.43:0/874084360 wait complete. 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 8 is available 2024-09-13T13:23:42.518 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying orchestrator module is enabled... 2024-09-13T13:23:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/197283898' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-13T13:23:43.761 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.903+0000 7fb5456ff640 1 Processor -- start 2024-09-13T13:23:43.761 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.903+0000 7fb5456ff640 1 -- start start 2024-09-13T13:23:43.761 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.903+0000 7fb5456ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb540109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.903+0000 7fb5456ff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401096c0 con 0x7fb540079bc0 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.904+0000 7fb53effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb540109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.904+0000 7fb53effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb540109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40744/0 (socket says 172.21.15.43:40744) 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.904+0000 7fb53effd640 1 -- 172.21.15.43:0/3769171137 learned_addr learned my addr 172.21.15.43:0/3769171137 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.904+0000 7fb53effd640 1 -- 172.21.15.43:0/3769171137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb540109840 con 0x7fb540079bc0 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.905+0000 7fb53effd640 1 --2- 172.21.15.43:0/3769171137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb540109180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb538009820 tx=0x7fb53802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ebe6e1e557bdf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.905+0000 7fb53dffb640 1 -- 172.21.15.43:0/3769171137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb538037070 con 0x7fb540079bc0 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.905+0000 7fb53dffb640 1 -- 172.21.15.43:0/3769171137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fb538032870 con 0x7fb540079bc0 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.905+0000 7fb53dffb640 1 -- 172.21.15.43:0/3769171137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb538030660 con 0x7fb540079bc0 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.905+0000 7fb5456ff640 1 -- 172.21.15.43:0/3769171137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 msgr2=0x7fb540109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.905+0000 7fb5456ff640 1 --2- 172.21.15.43:0/3769171137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb540109180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb538009820 tx=0x7fb53802a300 comp rx=0 tx=0).stop 2024-09-13T13:23:43.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.905+0000 7fb5456ff640 1 -- 172.21.15.43:0/3769171137 shutdown_connections 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.906+0000 7fb5456ff640 1 --2- 172.21.15.43:0/3769171137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb540109180 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.906+0000 7fb5456ff640 1 -- 172.21.15.43:0/3769171137 >> 172.21.15.43:0/3769171137 conn(0x7fb540100df0 msgr2=0x7fb540103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.906+0000 7fb5456ff640 1 -- 172.21.15.43:0/3769171137 shutdown_connections 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.906+0000 7fb5456ff640 1 -- 172.21.15.43:0/3769171137 wait complete. 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.906+0000 7fb5456ff640 1 Processor -- start 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.906+0000 7fb5456ff640 1 -- start start 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.906+0000 7fb5456ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb54019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb5456ff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb54010a8b0 con 0x7fb540079bc0 2024-09-13T13:23:43.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb53effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb54019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb53effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb54019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40758/0 (socket says 172.21.15.43:40758) 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb53effd640 1 -- 172.21.15.43:0/197283898 learned_addr learned my addr 172.21.15.43:0/197283898 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb53effd640 1 -- 172.21.15.43:0/197283898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54019abf0 con 0x7fb540079bc0 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb53effd640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb54019a690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb538030380 tx=0x7fb538030b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb538037040 con 0x7fb540079bc0 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fb538040070 con 0x7fb540079bc0 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54019ae80 con 0x7fb540079bc0 2024-09-13T13:23:43.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb53803a390 con 0x7fb540079bc0 2024-09-13T13:23:43.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.907+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54019b840 con 0x7fb540079bc0 2024-09-13T13:23:43.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.908+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 51765+0+0 (secure 0 0 0) 0x7fb53803b430 con 0x7fb540079bc0 2024-09-13T13:23:43.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.908+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5401071f0 con 0x7fb540079bc0 2024-09-13T13:23:43.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.909+0000 7fb537fff640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb52c03f6a0 0x7fb52c041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:43.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.909+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb5380741e0 con 0x7fb540079bc0 2024-09-13T13:23:43.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.909+0000 7fb53e7fc640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb52c03f6a0 0x7fb52c041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.909+0000 7fb53e7fc640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb52c03f6a0 0x7fb52c041b60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5280098c0 tx=0x7fb528006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:42.914+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb53803b720 con 0x7fb540079bc0 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.153+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "orchestrator"} v 0) -- 0x7fb54019c020 con 0x7fb540079bc0 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.240+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "orchestrator"}]=0 module 'orchestrator' is already enabled (always-on) v9) ==== 143+0+0 (secure 0 0 0) 0x7fb53803ab60 con 0x7fb540079bc0 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.241+0000 7fb537fff640 1 -- 172.21.15.43:0/197283898 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7fb53803a640 con 0x7fb540079bc0 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb52c03f6a0 msgr2=0x7fb52c041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb52c03f6a0 0x7fb52c041b60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5280098c0 tx=0x7fb528006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:43.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 msgr2=0x7fb54019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb54019a690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb538030380 tx=0x7fb538030b40 comp rx=0 tx=0).stop 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 shutdown_connections 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb52c03f6a0 0x7fb52c041b60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 --2- 172.21.15.43:0/197283898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb540079bc0 0x7fb54019a690 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 >> 172.21.15.43:0/197283898 conn(0x7fb540100df0 msgr2=0x7fb540101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 shutdown_connections 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:43.242+0000 7fb5456ff640 1 -- 172.21.15.43:0/197283898 wait complete. 2024-09-13T13:23:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-09-13T13:23:44.433 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/197283898' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-13T13:23:44.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:44 smithi043 ceph-mon[30106]: mgrmap e9: y(active, since 9s) 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.222+0000 7f8205611640 1 Processor -- start 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.222+0000 7f8205611640 1 -- start start 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.222+0000 7f8205611640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f82001091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.222+0000 7f8205611640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8200109720 con 0x7f8200079b60 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.223+0000 7f81feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f82001091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.223+0000 7f81feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f82001091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40764/0 (socket says 172.21.15.43:40764) 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.223+0000 7f81feffd640 1 -- 172.21.15.43:0/3631204865 learned_addr learned my addr 172.21.15.43:0/3631204865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.223+0000 7f81feffd640 1 -- 172.21.15.43:0/3631204865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82001098a0 con 0x7f8200079b60 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f81feffd640 1 --2- 172.21.15.43:0/3631204865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f82001091e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009e80 tx=0x7f81f002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec8c6326375e5cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:44.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f81fdffb640 1 -- 172.21.15.43:0/3631204865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81f0036070 con 0x7f8200079b60 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f81fdffb640 1 -- 172.21.15.43:0/3631204865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f81f0031830 con 0x7f8200079b60 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f81fdffb640 1 -- 172.21.15.43:0/3631204865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81f0007780 con 0x7f8200079b60 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f8205611640 1 -- 172.21.15.43:0/3631204865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 msgr2=0x7f82001091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f8205611640 1 --2- 172.21.15.43:0/3631204865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f82001091e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009e80 tx=0x7f81f002a7a0 comp rx=0 tx=0).stop 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f8205611640 1 -- 172.21.15.43:0/3631204865 shutdown_connections 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f8205611640 1 --2- 172.21.15.43:0/3631204865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f82001091e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.224+0000 7f8205611640 1 -- 172.21.15.43:0/3631204865 >> 172.21.15.43:0/3631204865 conn(0x7f8200100df0 msgr2=0x7f8200103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.225+0000 7f8205611640 1 -- 172.21.15.43:0/3631204865 shutdown_connections 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.225+0000 7f8205611640 1 -- 172.21.15.43:0/3631204865 wait complete. 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.225+0000 7f8205611640 1 Processor -- start 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.225+0000 7f8205611640 1 -- start start 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.225+0000 7f8205611640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f820019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:45.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.225+0000 7f8205611640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f820010a840 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f820019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f820019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40780/0 (socket says 172.21.15.43:40780) 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81feffd640 1 -- 172.21.15.43:0/1423435032 learned_addr learned my addr 172.21.15.43:0/1423435032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81feffd640 1 -- 172.21.15.43:0/1423435032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820019ab80 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81feffd640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f820019a620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009e50 tx=0x7f81f00077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81e7fff640 1 -- 172.21.15.43:0/1423435032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81f0041020 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81e7fff640 1 -- 172.21.15.43:0/1423435032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f81f0007cc0 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820019ae10 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f81e7fff640 1 -- 172.21.15.43:0/1423435032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81f0036040 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.226+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820019b850 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.227+0000 7f81e7fff640 1 -- 172.21.15.43:0/1423435032 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7f81f0044020 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.227+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8200107250 con 0x7f8200079b60 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.227+0000 7f81e7fff640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f81e803f6a0 0x7f81e8041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:45.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.228+0000 7f81e7fff640 1 -- 172.21.15.43:0/1423435032 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f81f0075160 con 0x7f8200079b60 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.228+0000 7f81fe7fc640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f81e803f6a0 0x7f81e8041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.228+0000 7f81fe7fc640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f81e803f6a0 0x7f81e8041b60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f81f40098c0 tx=0x7f81f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.232+0000 7f81e7fff640 1 -- 172.21.15.43:0/1423435032 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f81f004b020 con 0x7f8200079b60 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.432+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f820019c030 con 0x7f81e803f6a0 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.437+0000 7f81e7fff640 1 -- 172.21.15.43:0/1423435032 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f820019c030 con 0x7f81e803f6a0 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f81e803f6a0 msgr2=0x7f81e8041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f81e803f6a0 0x7f81e8041b60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f81f40098c0 tx=0x7f81f4006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 msgr2=0x7f820019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f820019a620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009e50 tx=0x7f81f00077a0 comp rx=0 tx=0).stop 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 shutdown_connections 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f81e803f6a0 0x7f81e8041b60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 --2- 172.21.15.43:0/1423435032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8200079b60 0x7f820019a620 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.439+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 >> 172.21.15.43:0/1423435032 conn(0x7f8200100df0 msgr2=0x7f8200101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:45.002 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.440+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 shutdown_connections 2024-09-13T13:23:45.003 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:44.440+0000 7f8205611640 1 -- 172.21.15.43:0/1423435032 wait complete. 2024-09-13T13:23:45.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:45 smithi043 ceph-mon[30106]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:23:45.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:45 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:45.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:45 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:23:46.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-13T13:23:46.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.402+0000 7fae09efa640 1 Processor -- start 2024-09-13T13:23:46.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.402+0000 7fae09efa640 1 -- start start 2024-09-13T13:23:46.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.402+0000 7fae09efa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae041091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:46.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.402+0000 7fae09efa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae04109700 con 0x7fae04079b80 2024-09-13T13:23:46.114 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.403+0000 7fae037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae041091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.403+0000 7fae037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae041091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40788/0 (socket says 172.21.15.43:40788) 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.403+0000 7fae037fe640 1 -- 172.21.15.43:0/2663295935 learned_addr learned my addr 172.21.15.43:0/2663295935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.403+0000 7fae037fe640 1 -- 172.21.15.43:0/2663295935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae04109880 con 0x7fae04079b80 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae037fe640 1 --2- 172.21.15.43:0/2663295935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae041091c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009e80 tx=0x7fadf402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbb56dd2b9bda3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae027fc640 1 -- 172.21.15.43:0/2663295935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf4036070 con 0x7fae04079b80 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae027fc640 1 -- 172.21.15.43:0/2663295935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fadf4031830 con 0x7fae04079b80 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae027fc640 1 -- 172.21.15.43:0/2663295935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf4007780 con 0x7fae04079b80 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae09efa640 1 -- 172.21.15.43:0/2663295935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 msgr2=0x7fae041091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae09efa640 1 --2- 172.21.15.43:0/2663295935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae041091c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009e80 tx=0x7fadf402a7a0 comp rx=0 tx=0).stop 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae09efa640 1 -- 172.21.15.43:0/2663295935 shutdown_connections 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae09efa640 1 --2- 172.21.15.43:0/2663295935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae041091c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:46.115 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.404+0000 7fae09efa640 1 -- 172.21.15.43:0/2663295935 >> 172.21.15.43:0/2663295935 conn(0x7fae04100df0 msgr2=0x7fae04103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.405+0000 7fae09efa640 1 -- 172.21.15.43:0/2663295935 shutdown_connections 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.405+0000 7fae09efa640 1 -- 172.21.15.43:0/2663295935 wait complete. 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.405+0000 7fae09efa640 1 Processor -- start 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.405+0000 7fae09efa640 1 -- start start 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.405+0000 7fae09efa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae0419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.405+0000 7fae09efa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae0410a8a0 con 0x7fae04079b80 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae0419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae0419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40804/0 (socket says 172.21.15.43:40804) 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae037fe640 1 -- 172.21.15.43:0/3816422084 learned_addr learned my addr 172.21.15.43:0/3816422084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae037fe640 1 -- 172.21.15.43:0/3816422084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0419cdb0 con 0x7fae04079b80 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae037fe640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae0419c850 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009fa0 tx=0x7fadf40077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae00ff9640 1 -- 172.21.15.43:0/3816422084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf4041020 con 0x7fae04079b80 2024-09-13T13:23:46.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae00ff9640 1 -- 172.21.15.43:0/3816422084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fadf403c470 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0419d040 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae00ff9640 1 -- 172.21.15.43:0/3816422084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf4036040 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.406+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0419da80 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.407+0000 7fae00ff9640 1 -- 172.21.15.43:0/3816422084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7fadf4044020 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.407+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae04107230 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.407+0000 7fae00ff9640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fadf003f6a0 0x7fadf0041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.408+0000 7fae00ff9640 1 -- 172.21.15.43:0/3816422084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fadf4075190 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.408+0000 7fae02ffd640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fadf003f6a0 0x7fadf0041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.408+0000 7fae02ffd640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fadf003f6a0 0x7fadf0041b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098c0 tx=0x7fadec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.412+0000 7fae00ff9640 1 -- 172.21.15.43:0/3816422084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fadf404b020 con 0x7fae04079b80 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.600+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fae0419e330 con 0x7fadf003f6a0 2024-09-13T13:23:46.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.600+0000 7fae00ff9640 1 -- 172.21.15.43:0/3816422084 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fae0419e330 con 0x7fadf003f6a0 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fadf003f6a0 msgr2=0x7fadf0041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fadf003f6a0 0x7fadf0041b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098c0 tx=0x7fadec006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 msgr2=0x7fae0419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae0419c850 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009fa0 tx=0x7fadf40077a0 comp rx=0 tx=0).stop 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 shutdown_connections 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fadf003f6a0 0x7fadf0041b60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 --2- 172.21.15.43:0/3816422084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae04079b80 0x7fae0419c850 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 >> 172.21.15.43:0/3816422084 conn(0x7fae04100df0 msgr2=0x7fae041019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 shutdown_connections 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:45.603+0000 7fae09efa640 1 -- 172.21.15.43:0/3816422084 wait complete. 2024-09-13T13:23:46.118 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-09-13T13:23:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-mon[30106]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:23:47.242 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: Generating public/private rsa key pair. 2024-09-13T13:23:47.242 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: Your identification has been saved in /tmp/tmphetc0dxo/key 2024-09-13T13:23:47.242 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: Your public key has been saved in /tmp/tmphetc0dxo/key.pub 2024-09-13T13:23:47.242 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: The key fingerprint is: 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: SHA256:iO4E/8EXp3/j9MNoDn4wfW3wQeQbWO2HBHZ+lX7op2E ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: The key's randomart image is: 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: +---[RSA 3072]----+ 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | o.oo+| 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | . o=oo| 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | o+*.| 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | . . ++*| 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | . . . S .. . =+| 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | + . +o . E =| 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | + o o .o.= = | 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | o . o o o*.+ | 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: | . . o*o... | 2024-09-13T13:23:47.243 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:23:46 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: +----[SHA256]-----+ 2024-09-13T13:23:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50dd271640 1 Processor -- start 2024-09-13T13:23:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50dd271640 1 -- start start 2024-09-13T13:23:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50dd271640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d8106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50dd271640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8107000 con 0x7f50d8106610 2024-09-13T13:23:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d8106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d8106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43506/0 (socket says 172.21.15.43:43506) 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50d6ffd640 1 -- 172.21.15.43:0/3266668580 learned_addr learned my addr 172.21.15.43:0/3266668580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.658+0000 7f50d6ffd640 1 -- 172.21.15.43:0/3266668580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d8107180 con 0x7f50d8106610 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.659+0000 7f50d6ffd640 1 --2- 172.21.15.43:0/3266668580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d8106a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009820 tx=0x7f50c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ef024abb462387 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.659+0000 7f50d5ffb640 1 -- 172.21.15.43:0/3266668580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50c8037070 con 0x7f50d8106610 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.659+0000 7f50d5ffb640 1 -- 172.21.15.43:0/3266668580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f50c8032870 con 0x7f50d8106610 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.659+0000 7f50d5ffb640 1 -- 172.21.15.43:0/3266668580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50c80305d0 con 0x7f50d8106610 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 -- 172.21.15.43:0/3266668580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 msgr2=0x7f50d8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 --2- 172.21.15.43:0/3266668580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d8106a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009820 tx=0x7f50c802a270 comp rx=0 tx=0).stop 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 -- 172.21.15.43:0/3266668580 shutdown_connections 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 --2- 172.21.15.43:0/3266668580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d8106a10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 -- 172.21.15.43:0/3266668580 >> 172.21.15.43:0/3266668580 conn(0x7f50d8101f00 msgr2=0x7f50d8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:47.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 -- 172.21.15.43:0/3266668580 shutdown_connections 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 -- 172.21.15.43:0/3266668580 wait complete. 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.660+0000 7f50dd271640 1 Processor -- start 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50dd271640 1 -- start start 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50dd271640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50dd271640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50d810aa90 con 0x7f50d8106610 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43510/0 (socket says 172.21.15.43:43510) 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50d6ffd640 1 -- 172.21.15.43:0/3448975605 learned_addr learned my addr 172.21.15.43:0/3448975605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50d6ffd640 1 -- 172.21.15.43:0/3448975605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d819abb0 con 0x7f50d8106610 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50d6ffd640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d819a650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50c80097f0 tx=0x7f50c8030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50cffff640 1 -- 172.21.15.43:0/3448975605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50c803f020 con 0x7f50d8106610 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50cffff640 1 -- 172.21.15.43:0/3448975605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f50c8030e40 con 0x7f50d8106610 2024-09-13T13:23:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.661+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d819ae40 con 0x7f50d8106610 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.662+0000 7f50cffff640 1 -- 172.21.15.43:0/3448975605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50c8037040 con 0x7f50d8106610 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.662+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d819b800 con 0x7f50d8106610 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.662+0000 7f50cffff640 1 -- 172.21.15.43:0/3448975605 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7f50c8042020 con 0x7f50d8106610 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.663+0000 7f50cffff640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f50c403f6a0 0x7f50c4041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.663+0000 7f50cffff640 1 -- 172.21.15.43:0/3448975605 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f50c8074540 con 0x7f50d8106610 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.663+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d810ac10 con 0x7f50d8106610 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.663+0000 7f50d67fc640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f50c403f6a0 0x7f50c4041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.663+0000 7f50d67fc640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f50c403f6a0 0x7f50c4041b60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50c00098c0 tx=0x7f50c0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.669+0000 7f50cffff640 1 -- 172.21.15.43:0/3448975605 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f50c8049020 con 0x7f50d8106610 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:46.855+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f50d81031e0 con 0x7f50c403f6a0 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.001+0000 7f50cffff640 1 -- 172.21.15.43:0/3448975605 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50d81031e0 con 0x7f50c403f6a0 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f50c403f6a0 msgr2=0x7f50c4041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f50c403f6a0 0x7f50c4041b60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50c00098c0 tx=0x7f50c0006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 msgr2=0x7f50d819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d819a650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50c80097f0 tx=0x7f50c8030b00 comp rx=0 tx=0).stop 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 shutdown_connections 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f50c403f6a0 0x7f50c4041b60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 --2- 172.21.15.43:0/3448975605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d8106610 0x7f50d819a650 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 >> 172.21.15.43:0/3448975605 conn(0x7f50d8101f00 msgr2=0x7f50d8102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 shutdown_connections 2024-09-13T13:23:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.003+0000 7f50dd271640 1 -- 172.21.15.43:0/3448975605 wait complete. 2024-09-13T13:23:48.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:47 smithi043 ceph-mon[30106]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:23:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:47 smithi043 ceph-mon[30106]: Generating ssh key... 2024-09-13T13:23:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:47 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:47 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjLkD3SbEjMgMtLUMLCxIB8Rn9GPtg9iA4GqWze4u2DQlo5xzPcR+l/xxvqff5eDNGIESRenYicsQlcSd/zP91blQWVBszIe2+PmuWVaGJsGjEOqqqiAVcpORpSA9PueZQ/YtAskbF5yTwRQoRLGxosgjzg2PpsyIQbOVpJN6ss7grn7Gfdyh2jVlcs2csSgjhEEa5yk9Nca1Njr/qKx7u+1RCA4tN0lJ4TiyAVIl3XnTDcxISxAKQQqMx6M7Pue84H7HqEkTURCstdpwPjWJBmWFS9US0vR9r5MCUIy+5ebwX/I8uXk4jQV50JJwMj1VtQFUMNi/uQX8YIKVDnbdd+GtXOzZSxRpbDaMLK+12VtMKPk8McBW5+ZOVlYLR/CX/laTPsfRdVUgnYvcAJaNzxKQh/0QvxgiRTAJ0pgSfa2vsi93OkrA4JvlPLwFqiv3xq4FHAQ886w5BPMx99gcym2QLB7C/s5nWmzhJXi84noSbSGxuy3ACctXrBt+LvT0= ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.972+0000 7fccd9f26640 1 Processor -- start 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.973+0000 7fccd9f26640 1 -- start start 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.973+0000 7fccd9f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd4109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.973+0000 7fccd9f26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41096c0 con 0x7fccd4079bc0 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.973+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd4109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.973+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd4109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43514/0 (socket says 172.21.15.43:43514) 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.973+0000 7fccd37fe640 1 -- 172.21.15.43:0/2819321480 learned_addr learned my addr 172.21.15.43:0/2819321480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:48.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.973+0000 7fccd37fe640 1 -- 172.21.15.43:0/2819321480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd4109840 con 0x7fccd4079bc0 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.974+0000 7fccd37fe640 1 --2- 172.21.15.43:0/2819321480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd4109180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009820 tx=0x7fccc402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb603ef2ea3d5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.974+0000 7fccd27fc640 1 -- 172.21.15.43:0/2819321480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc4037070 con 0x7fccd4079bc0 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.974+0000 7fccd27fc640 1 -- 172.21.15.43:0/2819321480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fccc4032870 con 0x7fccd4079bc0 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.974+0000 7fccd27fc640 1 -- 172.21.15.43:0/2819321480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc4030660 con 0x7fccd4079bc0 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 -- 172.21.15.43:0/2819321480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 msgr2=0x7fccd4109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 --2- 172.21.15.43:0/2819321480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd4109180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009820 tx=0x7fccc402a300 comp rx=0 tx=0).stop 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 -- 172.21.15.43:0/2819321480 shutdown_connections 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 --2- 172.21.15.43:0/2819321480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd4109180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 -- 172.21.15.43:0/2819321480 >> 172.21.15.43:0/2819321480 conn(0x7fccd4100df0 msgr2=0x7fccd4103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 -- 172.21.15.43:0/2819321480 shutdown_connections 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 -- 172.21.15.43:0/2819321480 wait complete. 2024-09-13T13:23:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.975+0000 7fccd9f26640 1 Processor -- start 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd9f26640 1 -- start start 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd9f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd41067d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd9f26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410a8b0 con 0x7fccd4079bc0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd41067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd41067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43518/0 (socket says 172.21.15.43:43518) 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd37fe640 1 -- 172.21.15.43:0/1622568393 learned_addr learned my addr 172.21.15.43:0/1622568393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd37fe640 1 -- 172.21.15.43:0/1622568393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd4106d30 con 0x7fccd4079bc0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd37fe640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd41067d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fccc4000c00 tx=0x7fccc4030a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd0ff9640 1 -- 172.21.15.43:0/1622568393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc4037040 con 0x7fccd4079bc0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.976+0000 7fccd0ff9640 1 -- 172.21.15.43:0/1622568393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fccc4040070 con 0x7fccd4079bc0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.977+0000 7fccd0ff9640 1 -- 172.21.15.43:0/1622568393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc403a390 con 0x7fccd4079bc0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.977+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd4105120 con 0x7fccd4079bc0 2024-09-13T13:23:48.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.977+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41056a0 con 0x7fccd4079bc0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.978+0000 7fccd0ff9640 1 -- 172.21.15.43:0/1622568393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7fccc403b430 con 0x7fccd4079bc0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.978+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd41071f0 con 0x7fccd4079bc0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.978+0000 7fccd0ff9640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fccbc03f6a0 0x7fccbc041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.978+0000 7fccd0ff9640 1 -- 172.21.15.43:0/1622568393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fccc4074210 con 0x7fccd4079bc0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.978+0000 7fccd2ffd640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fccbc03f6a0 0x7fccbc041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.978+0000 7fccd2ffd640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fccbc03f6a0 0x7fccbc041b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccc80098c0 tx=0x7fccc8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:47.983+0000 7fccd0ff9640 1 -- 172.21.15.43:0/1622568393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fccc402e200 con 0x7fccd4079bc0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.169+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fccd4105cd0 con 0x7fccbc03f6a0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.169+0000 7fccd0ff9640 1 -- 172.21.15.43:0/1622568393 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fccd4105cd0 con 0x7fccbc03f6a0 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fccbc03f6a0 msgr2=0x7fccbc041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fccbc03f6a0 0x7fccbc041b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccc80098c0 tx=0x7fccc8006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:48.740 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 msgr2=0x7fccd41067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd41067d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fccc4000c00 tx=0x7fccc4030a60 comp rx=0 tx=0).stop 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 shutdown_connections 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fccbc03f6a0 0x7fccbc041b60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 --2- 172.21.15.43:0/1622568393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccd4079bc0 0x7fccd41067d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 >> 172.21.15.43:0/1622568393 conn(0x7fccd4100df0 msgr2=0x7fccd4101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 shutdown_connections 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:48.172+0000 7fccd9f26640 1 -- 172.21.15.43:0/1622568393 wait complete. 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-09-13T13:23:48.741 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-09-13T13:23:49.098 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:48 smithi043 ceph-mon[30106]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:23:50.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:50 smithi043 ceph-mon[30106]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:23:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:51 smithi043 ceph-mon[30106]: Deploying cephadm binary to smithi043 2024-09-13T13:23:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-09-13T13:23:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.182+0000 7f55df375640 1 Processor -- start 2024-09-13T13:23:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.182+0000 7f55df375640 1 -- start start 2024-09-13T13:23:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.182+0000 7f55df375640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d807ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.182+0000 7f55df375640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55d807c9a0 con 0x7f55d807c4f0 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.182+0000 7f55dd111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d807ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.182+0000 7f55dd111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d807ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43530/0 (socket says 172.21.15.43:43530) 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.182+0000 7f55dd111640 1 -- 172.21.15.43:0/3464845743 learned_addr learned my addr 172.21.15.43:0/3464845743 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.183+0000 7f55dd111640 1 -- 172.21.15.43:0/3464845743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d807b090 con 0x7f55d807c4f0 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.183+0000 7f55dd111640 1 --2- 172.21.15.43:0/3464845743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d807ab50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009820 tx=0x7f55d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6574db141fe44018 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55cffff640 1 -- 172.21.15.43:0/3464845743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d4037070 con 0x7f55d807c4f0 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55cffff640 1 -- 172.21.15.43:0/3464845743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f55d4032870 con 0x7f55d807c4f0 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55cffff640 1 -- 172.21.15.43:0/3464845743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d40305d0 con 0x7f55d807c4f0 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55df375640 1 -- 172.21.15.43:0/3464845743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 msgr2=0x7f55d807ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55df375640 1 --2- 172.21.15.43:0/3464845743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d807ab50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009820 tx=0x7f55d402a270 comp rx=0 tx=0).stop 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55df375640 1 -- 172.21.15.43:0/3464845743 shutdown_connections 2024-09-13T13:23:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55df375640 1 --2- 172.21.15.43:0/3464845743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d807ab50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55df375640 1 -- 172.21.15.43:0/3464845743 >> 172.21.15.43:0/3464845743 conn(0x7f55d8101f00 msgr2=0x7f55d8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55df375640 1 -- 172.21.15.43:0/3464845743 shutdown_connections 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.184+0000 7f55df375640 1 -- 172.21.15.43:0/3464845743 wait complete. 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.185+0000 7f55df375640 1 Processor -- start 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.185+0000 7f55df375640 1 -- start start 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.185+0000 7f55df375640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.185+0000 7f55df375640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55d810cb30 con 0x7f55d807c4f0 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.185+0000 7f55dd111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.185+0000 7f55dd111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43532/0 (socket says 172.21.15.43:43532) 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.185+0000 7f55dd111640 1 -- 172.21.15.43:0/1056326561 learned_addr learned my addr 172.21.15.43:0/1056326561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.186+0000 7f55dd111640 1 -- 172.21.15.43:0/1056326561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d819cdb0 con 0x7f55d807c4f0 2024-09-13T13:23:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.186+0000 7f55dd111640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d819c850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f55d40090b0 tx=0x7f55d40307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.186+0000 7f55ce7fc640 1 -- 172.21.15.43:0/1056326561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d403f020 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.186+0000 7f55ce7fc640 1 -- 172.21.15.43:0/1056326561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f55d4030e20 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.186+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d819d040 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.186+0000 7f55ce7fc640 1 -- 172.21.15.43:0/1056326561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d4037040 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.186+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d819da00 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.187+0000 7f55ce7fc640 1 -- 172.21.15.43:0/1056326561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7f55d4042020 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.187+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d807abd0 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.187+0000 7f55ce7fc640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f55c403f6a0 0x7f55c4041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.187+0000 7f55ce7fc640 1 -- 172.21.15.43:0/1056326561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f55d4074610 con 0x7f55d807c4f0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.188+0000 7f55dc910640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f55c403f6a0 0x7f55c4041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.188+0000 7f55dc910640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f55c403f6a0 0x7f55c4041b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f55c80098c0 tx=0x7f55c8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.192+0000 7f55ce7fc640 1 -- 172.21.15.43:0/1056326561 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f55d4049020 con 0x7f55d807c4f0 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:49.386+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) -- 0x7f55d819e250 con 0x7f55c403f6a0 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.176+0000 7f55ce7fc640 1 -- 172.21.15.43:0/1056326561 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f55d819e250 con 0x7f55c403f6a0 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f55c403f6a0 msgr2=0x7f55c4041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f55c403f6a0 0x7f55c4041b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f55c80098c0 tx=0x7f55c8006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 msgr2=0x7f55d819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d819c850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f55d40090b0 tx=0x7f55d40307c0 comp rx=0 tx=0).stop 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 shutdown_connections 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f55c403f6a0 0x7f55c4041b60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 --2- 172.21.15.43:0/1056326561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d807c4f0 0x7f55d819c850 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.179+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 >> 172.21.15.43:0/1056326561 conn(0x7f55d8101f00 msgr2=0x7f55d8102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.180+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 shutdown_connections 2024-09-13T13:23:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:53.180+0000 7f55df375640 1 -- 172.21.15.43:0/1056326561 wait complete. 2024-09-13T13:23:53.791 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-09-13T13:23:54.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:54 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:54.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:54 smithi043 ceph-mon[30106]: Added host smithi043 2024-09-13T13:23:54.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:54 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.203+0000 7ff5fbb25640 1 Processor -- start 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.203+0000 7ff5fbb25640 1 -- start start 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.203+0000 7ff5fbb25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f407ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.203+0000 7ff5fbb25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f407c9a0 con 0x7ff5f407c4f0 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.204+0000 7ff5f98c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f407ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.204+0000 7ff5f98c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f407ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43534/0 (socket says 172.21.15.43:43534) 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.204+0000 7ff5f98c1640 1 -- 172.21.15.43:0/2034010832 learned_addr learned my addr 172.21.15.43:0/2034010832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:55.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.204+0000 7ff5f98c1640 1 -- 172.21.15.43:0/2034010832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f407b090 con 0x7ff5f407c4f0 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5f98c1640 1 --2- 172.21.15.43:0/2034010832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f407ab50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009820 tx=0x7ff5ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16fe5a4ecdf486d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5f88bf640 1 -- 172.21.15.43:0/2034010832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec037070 con 0x7ff5f407c4f0 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5f88bf640 1 -- 172.21.15.43:0/2034010832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7ff5ec032870 con 0x7ff5f407c4f0 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5f88bf640 1 -- 172.21.15.43:0/2034010832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec0305d0 con 0x7ff5f407c4f0 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5fbb25640 1 -- 172.21.15.43:0/2034010832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 msgr2=0x7ff5f407ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5fbb25640 1 --2- 172.21.15.43:0/2034010832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f407ab50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009820 tx=0x7ff5ec02a270 comp rx=0 tx=0).stop 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5fbb25640 1 -- 172.21.15.43:0/2034010832 shutdown_connections 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5fbb25640 1 --2- 172.21.15.43:0/2034010832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f407ab50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5fbb25640 1 -- 172.21.15.43:0/2034010832 >> 172.21.15.43:0/2034010832 conn(0x7ff5f4101f00 msgr2=0x7ff5f4104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.205+0000 7ff5fbb25640 1 -- 172.21.15.43:0/2034010832 shutdown_connections 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5fbb25640 1 -- 172.21.15.43:0/2034010832 wait complete. 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5fbb25640 1 Processor -- start 2024-09-13T13:23:55.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5fbb25640 1 -- start start 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5fbb25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5fbb25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f410cb30 con 0x7ff5f407c4f0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5f98c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5f98c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43542/0 (socket says 172.21.15.43:43542) 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.206+0000 7ff5f98c1640 1 -- 172.21.15.43:0/1802939835 learned_addr learned my addr 172.21.15.43:0/1802939835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.207+0000 7ff5f98c1640 1 -- 172.21.15.43:0/1802939835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f419cdb0 con 0x7ff5f407c4f0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.207+0000 7ff5f98c1640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f419c850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097f0 tx=0x7ff5ec030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.207+0000 7ff5f2ffd640 1 -- 172.21.15.43:0/1802939835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec03f020 con 0x7ff5f407c4f0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.207+0000 7ff5f2ffd640 1 -- 172.21.15.43:0/1802939835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7ff5ec030e40 con 0x7ff5f407c4f0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.207+0000 7ff5f2ffd640 1 -- 172.21.15.43:0/1802939835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec037040 con 0x7ff5f407c4f0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.207+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f419d040 con 0x7ff5f407c4f0 2024-09-13T13:23:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.207+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f419da00 con 0x7ff5f407c4f0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.208+0000 7ff5f2ffd640 1 -- 172.21.15.43:0/1802939835 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7ff5ec042020 con 0x7ff5f407c4f0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.208+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f407abd0 con 0x7ff5f407c4f0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.208+0000 7ff5f2ffd640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7ff5dc03f6a0 0x7ff5dc041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.208+0000 7ff5f2ffd640 1 -- 172.21.15.43:0/1802939835 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5ec074500 con 0x7ff5f407c4f0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.209+0000 7ff5f90c0640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7ff5dc03f6a0 0x7ff5dc041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.209+0000 7ff5f90c0640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7ff5dc03f6a0 0x7ff5dc041b60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098c0 tx=0x7ff5e8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.213+0000 7ff5f2ffd640 1 -- 172.21.15.43:0/1802939835 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff5ec049020 con 0x7ff5f407c4f0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.438+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff5f41031f0 con 0x7ff5dc03f6a0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.442+0000 7ff5f2ffd640 1 -- 172.21.15.43:0/1802939835 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff5f41031f0 con 0x7ff5dc03f6a0 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7ff5dc03f6a0 msgr2=0x7ff5dc041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7ff5dc03f6a0 0x7ff5dc041b60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098c0 tx=0x7ff5e8006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 msgr2=0x7ff5f419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f419c850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097f0 tx=0x7ff5ec030b00 comp rx=0 tx=0).stop 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 shutdown_connections 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7ff5dc03f6a0 0x7ff5dc041b60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 --2- 172.21.15.43:0/1802939835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f407c4f0 0x7ff5f419c850 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 >> 172.21.15.43:0/1802939835 conn(0x7ff5f4101f00 msgr2=0x7ff5f4102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 shutdown_connections 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:54.445+0000 7ff5fbb25640 1 -- 172.21.15.43:0/1802939835 wait complete. 2024-09-13T13:23:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-09-13T13:23:55.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:55 smithi043 ceph-mon[30106]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:23:55.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:55 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:56.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:56 smithi043 ceph-mon[30106]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:23:56.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:56 smithi043 ceph-mon[30106]: Saving service mon spec with placement count:5 2024-09-13T13:23:56.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:56 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:57.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-13T13:23:57.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.792+0000 7f75c0b83640 1 Processor -- start 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.792+0000 7f75c0b83640 1 -- start start 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.792+0000 7f75c0b83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.792+0000 7f75c0b83640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc107000 con 0x7f75bc106610 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.792+0000 7f75ba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.792+0000 7f75ba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43544/0 (socket says 172.21.15.43:43544) 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.792+0000 7f75ba59c640 1 -- 172.21.15.43:0/2177607740 learned_addr learned my addr 172.21.15.43:0/2177607740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.793+0000 7f75ba59c640 1 -- 172.21.15.43:0/2177607740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc107180 con 0x7f75bc106610 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.793+0000 7f75ba59c640 1 --2- 172.21.15.43:0/2177607740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc106a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009820 tx=0x7f75ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45879db07ee906a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.793+0000 7f75b959a640 1 -- 172.21.15.43:0/2177607740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75ac037070 con 0x7f75bc106610 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.793+0000 7f75b959a640 1 -- 172.21.15.43:0/2177607740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f75ac032870 con 0x7f75bc106610 2024-09-13T13:23:57.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.793+0000 7f75b959a640 1 -- 172.21.15.43:0/2177607740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75ac0305d0 con 0x7f75bc106610 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.794+0000 7f75c0b83640 1 -- 172.21.15.43:0/2177607740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 msgr2=0x7f75bc106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.794+0000 7f75c0b83640 1 --2- 172.21.15.43:0/2177607740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc106a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009820 tx=0x7f75ac02a270 comp rx=0 tx=0).stop 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.794+0000 7f75c0b83640 1 -- 172.21.15.43:0/2177607740 shutdown_connections 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.794+0000 7f75c0b83640 1 --2- 172.21.15.43:0/2177607740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc106a10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.794+0000 7f75c0b83640 1 -- 172.21.15.43:0/2177607740 >> 172.21.15.43:0/2177607740 conn(0x7f75bc101f00 msgr2=0x7f75bc104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.794+0000 7f75c0b83640 1 -- 172.21.15.43:0/2177607740 shutdown_connections 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.794+0000 7f75c0b83640 1 -- 172.21.15.43:0/2177607740 wait complete. 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75c0b83640 1 Processor -- start 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75c0b83640 1 -- start start 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75c0b83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75c0b83640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc10aa90 con 0x7f75bc106610 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75ba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:57.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75ba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43550/0 (socket says 172.21.15.43:43550) 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75ba59c640 1 -- 172.21.15.43:0/3619453997 learned_addr learned my addr 172.21.15.43:0/3619453997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.795+0000 7f75ba59c640 1 -- 172.21.15.43:0/3619453997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc19abb0 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.796+0000 7f75ba59c640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc19a650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0090b0 tx=0x7f75ac0307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.796+0000 7f75ab7fe640 1 -- 172.21.15.43:0/3619453997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75ac03f020 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.796+0000 7f75ab7fe640 1 -- 172.21.15.43:0/3619453997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f75ac030e20 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.796+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc19ae40 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.796+0000 7f75ab7fe640 1 -- 172.21.15.43:0/3619453997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75ac037040 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.796+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc19b800 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.797+0000 7f75ab7fe640 1 -- 172.21.15.43:0/3619453997 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7f75ac042020 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.797+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75bc10ac10 con 0x7f75bc106610 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.797+0000 7f75ab7fe640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f75a403f6a0 0x7f75a4041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:57.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.797+0000 7f75ab7fe640 1 -- 172.21.15.43:0/3619453997 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f75ac0745d0 con 0x7f75bc106610 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.797+0000 7f75b9d9b640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f75a403f6a0 0x7f75a4041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.798+0000 7f75b9d9b640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f75a403f6a0 0x7f75a4041b60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098c0 tx=0x7f75b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.802+0000 7f75ab7fe640 1 -- 172.21.15.43:0/3619453997 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f75ac049020 con 0x7f75bc106610 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:55.999+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f75bc1031e0 con 0x7f75a403f6a0 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.004+0000 7f75ab7fe640 1 -- 172.21.15.43:0/3619453997 <== mgr.14118 v2:172.21.15.43:6800/85481396 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f75bc1031e0 con 0x7f75a403f6a0 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f75a403f6a0 msgr2=0x7f75a4041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f75a403f6a0 0x7f75a4041b60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098c0 tx=0x7f75b0006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 msgr2=0x7f75bc19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc19a650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0090b0 tx=0x7f75ac0307c0 comp rx=0 tx=0).stop 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 shutdown_connections 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f75a403f6a0 0x7f75a4041b60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:57.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 --2- 172.21.15.43:0/3619453997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75bc106610 0x7f75bc19a650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:57.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 >> 172.21.15.43:0/3619453997 conn(0x7f75bc101f00 msgr2=0x7f75bc102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:57.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 shutdown_connections 2024-09-13T13:23:57.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:56.006+0000 7f75c0b83640 1 -- 172.21.15.43:0/3619453997 wait complete. 2024-09-13T13:23:57.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:57 smithi043 ceph-mon[30106]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:23:57.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:57 smithi043 ceph-mon[30106]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:23:57.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:57 smithi043 ceph-mon[30106]: Saving service mgr spec with placement count:2 2024-09-13T13:23:57.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:57 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:57.479 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:57 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.562+0000 7f3ed1f7b640 1 Processor -- start 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.562+0000 7f3ed1f7b640 1 -- start start 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.562+0000 7f3ed1f7b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc1058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.562+0000 7f3ed1f7b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc105eb0 con 0x7f3ecc1054c0 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.563+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc1058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.563+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc1058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34900/0 (socket says 172.21.15.43:34900) 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.563+0000 7f3ecb7fe640 1 -- 172.21.15.43:0/723315605 learned_addr learned my addr 172.21.15.43:0/723315605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.563+0000 7f3ecb7fe640 1 -- 172.21.15.43:0/723315605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc106030 con 0x7f3ecc1054c0 2024-09-13T13:23:58.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.564+0000 7f3ecb7fe640 1 --2- 172.21.15.43:0/723315605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc1058c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009e80 tx=0x7f3ebc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f3d8b02a5fb030 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.564+0000 7f3eca7fc640 1 -- 172.21.15.43:0/723315605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ebc036070 con 0x7f3ecc1054c0 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.564+0000 7f3eca7fc640 1 -- 172.21.15.43:0/723315605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3ebc031850 con 0x7f3ecc1054c0 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.564+0000 7f3eca7fc640 1 -- 172.21.15.43:0/723315605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ebc0076d0 con 0x7f3ecc1054c0 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.564+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/723315605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 msgr2=0x7f3ecc1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.564+0000 7f3ed1f7b640 1 --2- 172.21.15.43:0/723315605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc1058c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009e80 tx=0x7f3ebc02a7a0 comp rx=0 tx=0).stop 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.565+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/723315605 shutdown_connections 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.565+0000 7f3ed1f7b640 1 --2- 172.21.15.43:0/723315605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc1058c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.565+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/723315605 >> 172.21.15.43:0/723315605 conn(0x7f3ecc100df0 msgr2=0x7f3ecc103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.565+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/723315605 shutdown_connections 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.565+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/723315605 wait complete. 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.565+0000 7f3ed1f7b640 1 Processor -- start 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.565+0000 7f3ed1f7b640 1 -- start start 2024-09-13T13:23:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ed1f7b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc0797f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ed1f7b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc109940 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc0797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc0797f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34916/0 (socket says 172.21.15.43:34916) 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ecb7fe640 1 -- 172.21.15.43:0/980022193 learned_addr learned my addr 172.21.15.43:0/980022193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ecb7fe640 1 -- 172.21.15.43:0/980022193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc079d50 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ecb7fe640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc0797f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009680 tx=0x7f3ebc0316a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ec8ff9640 1 -- 172.21.15.43:0/980022193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ebc041020 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ec8ff9640 1 -- 172.21.15.43:0/980022193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3ebc03c470 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ec8ff9640 1 -- 172.21.15.43:0/980022193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ebc036040 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc19d2e0 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.566+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc19d860 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.567+0000 7f3ec8ff9640 1 -- 172.21.15.43:0/980022193 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7f3ebc044020 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.568+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc109ac0 con 0x7f3ecc1054c0 2024-09-13T13:23:58.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.568+0000 7f3ec8ff9640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f3eb803f6a0 0x7f3eb8041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.568+0000 7f3ec8ff9640 1 -- 172.21.15.43:0/980022193 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ebc0750b0 con 0x7f3ecc1054c0 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.568+0000 7f3ecaffd640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f3eb803f6a0 0x7f3eb8041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.568+0000 7f3ecaffd640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f3eb803f6a0 0x7f3eb8041b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098c0 tx=0x7f3eb4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.573+0000 7f3ec8ff9640 1 -- 172.21.15.43:0/980022193 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3ebc04b020 con 0x7f3ecc1054c0 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.768+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3ecc1058c0 con 0x7f3ecc1054c0 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.770+0000 7f3ec8ff9640 1 -- 172.21.15.43:0/980022193 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3ebc045c80 con 0x7f3ecc1054c0 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f3eb803f6a0 msgr2=0x7f3eb8041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f3eb803f6a0 0x7f3eb8041b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098c0 tx=0x7f3eb4006db0 comp rx=0 tx=0).stop 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 msgr2=0x7f3ecc0797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:23:58.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc0797f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009680 tx=0x7f3ebc0316a0 comp rx=0 tx=0).stop 2024-09-13T13:23:58.537 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 shutdown_connections 2024-09-13T13:23:58.537 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f3eb803f6a0 0x7f3eb8041b60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:58.537 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 --2- 172.21.15.43:0/980022193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc1054c0 0x7f3ecc0797f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:23:58.537 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 >> 172.21.15.43:0/980022193 conn(0x7f3ecc100df0 msgr2=0x7f3ecc101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:23:58.537 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 shutdown_connections 2024-09-13T13:23:58.537 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:57.773+0000 7f3ed1f7b640 1 -- 172.21.15.43:0/980022193 wait complete. 2024-09-13T13:23:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:58 smithi043 ceph-mon[30106]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:23:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:23:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/980022193' entity='client.admin' 2024-09-13T13:24:00.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb982bde640 1 Processor -- start 2024-09-13T13:24:00.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb982bde640 1 -- start start 2024-09-13T13:24:00.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb982bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c1058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:00.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb982bde640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c105eb0 con 0x7fb97c1054c0 2024-09-13T13:24:00.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb98097a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c1058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb98097a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c1058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34930/0 (socket says 172.21.15.43:34930) 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb98097a640 1 -- 172.21.15.43:0/1619099350 learned_addr learned my addr 172.21.15.43:0/1619099350 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.275+0000 7fb98097a640 1 -- 172.21.15.43:0/1619099350 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c106030 con 0x7fb97c1054c0 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.276+0000 7fb98097a640 1 --2- 172.21.15.43:0/1619099350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c1058c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb964009820 tx=0x7fb96402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14a3deafd288e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.276+0000 7fb97b7fe640 1 -- 172.21.15.43:0/1619099350 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb964037070 con 0x7fb97c1054c0 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.276+0000 7fb97b7fe640 1 -- 172.21.15.43:0/1619099350 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb964032890 con 0x7fb97c1054c0 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.276+0000 7fb97b7fe640 1 -- 172.21.15.43:0/1619099350 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb964030550 con 0x7fb97c1054c0 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.277+0000 7fb982bde640 1 -- 172.21.15.43:0/1619099350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 msgr2=0x7fb97c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.277+0000 7fb982bde640 1 --2- 172.21.15.43:0/1619099350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c1058c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb964009820 tx=0x7fb96402a270 comp rx=0 tx=0).stop 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.277+0000 7fb982bde640 1 -- 172.21.15.43:0/1619099350 shutdown_connections 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.277+0000 7fb982bde640 1 --2- 172.21.15.43:0/1619099350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c1058c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.277+0000 7fb982bde640 1 -- 172.21.15.43:0/1619099350 >> 172.21.15.43:0/1619099350 conn(0x7fb97c100df0 msgr2=0x7fb97c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.277+0000 7fb982bde640 1 -- 172.21.15.43:0/1619099350 shutdown_connections 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.277+0000 7fb982bde640 1 -- 172.21.15.43:0/1619099350 wait complete. 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb982bde640 1 Processor -- start 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb982bde640 1 -- start start 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb982bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb982bde640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c109940 con 0x7fb97c1054c0 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb98097a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb98097a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34944/0 (socket says 172.21.15.43:34944) 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb98097a640 1 -- 172.21.15.43:0/2672526784 learned_addr learned my addr 172.21.15.43:0/2672526784 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb98097a640 1 -- 172.21.15.43:0/2672526784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c19aba0 con 0x7fb97c1054c0 2024-09-13T13:24:00.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb98097a640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c19a640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097f0 tx=0x7fb964030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.278+0000 7fb979ffb640 1 -- 172.21.15.43:0/2672526784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb964037040 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.279+0000 7fb979ffb640 1 -- 172.21.15.43:0/2672526784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb964040070 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.279+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c19ae30 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.279+0000 7fb979ffb640 1 -- 172.21.15.43:0/2672526784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb96403bd00 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.279+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c19b7f0 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.280+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97c109ac0 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.280+0000 7fb979ffb640 1 -- 172.21.15.43:0/2672526784 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7fb96403b430 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.280+0000 7fb979ffb640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb96803f650 0x7fb968041b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.280+0000 7fb979ffb640 1 -- 172.21.15.43:0/2672526784 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb964074260 con 0x7fb97c1054c0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.280+0000 7fb97bfff640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb96803f650 0x7fb968041b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:00.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.281+0000 7fb97bfff640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb96803f650 0x7fb968041b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00ad30 tx=0x7fb96c0093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.285+0000 7fb979ffb640 1 -- 172.21.15.43:0/2672526784 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb9640313a0 con 0x7fb97c1054c0 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.486+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb97c1058c0 con 0x7fb97c1054c0 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.489+0000 7fb979ffb640 1 -- 172.21.15.43:0/2672526784 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb964030c60 con 0x7fb97c1054c0 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb96803f650 msgr2=0x7fb968041b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb96803f650 0x7fb968041b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00ad30 tx=0x7fb96c0093f0 comp rx=0 tx=0).stop 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 msgr2=0x7fb97c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c19a640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097f0 tx=0x7fb964030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 shutdown_connections 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fb96803f650 0x7fb968041b10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 --2- 172.21.15.43:0/2672526784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb97c1054c0 0x7fb97c19a640 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 >> 172.21.15.43:0/2672526784 conn(0x7fb97c100df0 msgr2=0x7fb97c078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 shutdown_connections 2024-09-13T13:24:00.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:23:59.492+0000 7fb982bde640 1 -- 172.21.15.43:0/2672526784 wait complete. 2024-09-13T13:24:00.114 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-09-13T13:24:00.368 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:00 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:24:00.368 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:00 smithi043 ceph-mon[30106]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:00.368 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2672526784' entity='client.admin' 2024-09-13T13:24:02.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:01 smithi043 ceph-mon[30106]: from='mgr.14118 172.21.15.43:0/322467154' entity='mgr.y' 2024-09-13T13:24:02.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3503796186' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-13T13:24:02.136 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:01 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ignoring --setuser ceph since I am not root 2024-09-13T13:24:02.136 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:01 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ignoring --setgroup ceph since I am not root 2024-09-13T13:24:02.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.581+0000 7fe530e18640 1 Processor -- start 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.581+0000 7fe530e18640 1 -- start start 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.581+0000 7fe530e18640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c10b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.581+0000 7fe530e18640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c079eb0 con 0x7fe52c079ab0 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.582+0000 7fe52a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c10b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.582+0000 7fe52a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c10b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34958/0 (socket says 172.21.15.43:34958) 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.582+0000 7fe52a59c640 1 -- 172.21.15.43:0/1943701189 learned_addr learned my addr 172.21.15.43:0/1943701189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.582+0000 7fe52a59c640 1 -- 172.21.15.43:0/1943701189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c10bac0 con 0x7fe52c079ab0 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.583+0000 7fe52a59c640 1 --2- 172.21.15.43:0/1943701189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c10b580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009820 tx=0x7fe51c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29808577c37368e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.583+0000 7fe529d9b640 1 -- 172.21.15.43:0/1943701189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c037070 con 0x7fe52c079ab0 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.583+0000 7fe529d9b640 1 -- 172.21.15.43:0/1943701189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fe51c032890 con 0x7fe52c079ab0 2024-09-13T13:24:02.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.583+0000 7fe529d9b640 1 -- 172.21.15.43:0/1943701189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c030550 con 0x7fe52c079ab0 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.584+0000 7fe530e18640 1 -- 172.21.15.43:0/1943701189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 msgr2=0x7fe52c10b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.584+0000 7fe530e18640 1 --2- 172.21.15.43:0/1943701189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c10b580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009820 tx=0x7fe51c02a270 comp rx=0 tx=0).stop 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.584+0000 7fe530e18640 1 -- 172.21.15.43:0/1943701189 shutdown_connections 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.584+0000 7fe530e18640 1 --2- 172.21.15.43:0/1943701189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c10b580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.584+0000 7fe530e18640 1 -- 172.21.15.43:0/1943701189 >> 172.21.15.43:0/1943701189 conn(0x7fe52c100df0 msgr2=0x7fe52c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.584+0000 7fe530e18640 1 -- 172.21.15.43:0/1943701189 shutdown_connections 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.584+0000 7fe530e18640 1 -- 172.21.15.43:0/1943701189 wait complete. 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.585+0000 7fe530e18640 1 Processor -- start 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.585+0000 7fe530e18640 1 -- start start 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.585+0000 7fe530e18640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.585+0000 7fe530e18640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c10cb20 con 0x7fe52c079ab0 2024-09-13T13:24:02.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.585+0000 7fe52a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.585+0000 7fe52a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34974/0 (socket says 172.21.15.43:34974) 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.585+0000 7fe52a59c640 1 -- 172.21.15.43:0/3503796186 learned_addr learned my addr 172.21.15.43:0/3503796186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.586+0000 7fe52a59c640 1 -- 172.21.15.43:0/3503796186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c19cd90 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.586+0000 7fe52a59c640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c19c830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0097f0 tx=0x7fe51c030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.586+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c037040 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.586+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fe51c040070 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.586+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c19d020 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.586+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c03bd00 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.586+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c19d9e0 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.587+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 51765+0+0 (secure 0 0 0) 0x7fe51c045460 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.587+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52c10b580 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.588+0000 7fe5237fe640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fe51803f6a0 0x7fe518041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.588+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe51c074170 con 0x7fe52c079ab0 2024-09-13T13:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.588+0000 7fe523fff640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fe51803f6a0 0x7fe518041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.588+0000 7fe523fff640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fe51803f6a0 0x7fe518041b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe5140098c0 tx=0x7fe514006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.592+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fe51c030c60 con 0x7fe52c079ab0 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:00.832+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fe52c19e120 con 0x7fe52c079ab0 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.799+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v10) ==== 88+0+0 (secure 0 0 0) 0x7fe51c0443c0 con 0x7fe52c079ab0 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.801+0000 7fe5237fe640 1 -- 172.21.15.43:0/3503796186 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 10) ==== 51778+0+0 (secure 0 0 0) 0x7fe51c041b40 con 0x7fe52c079ab0 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fe51803f6a0 msgr2=0x7fe518041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fe51803f6a0 0x7fe518041b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe5140098c0 tx=0x7fe514006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 msgr2=0x7fe52c19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c19c830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0097f0 tx=0x7fe51c030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 shutdown_connections 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7fe51803f6a0 0x7fe518041b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:02.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 --2- 172.21.15.43:0/3503796186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52c079ab0 0x7fe52c19c830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:02.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 >> 172.21.15.43:0/3503796186 conn(0x7fe52c100df0 msgr2=0x7fe52c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:02.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 shutdown_connections 2024-09-13T13:24:02.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:01.803+0000 7fe530e18640 1 -- 172.21.15.43:0/3503796186 wait complete. 2024-09-13T13:24:03.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3503796186' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-13T13:24:03.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:02 smithi043 ceph-mon[30106]: mgrmap e10: y(active, since 28s) 2024-09-13T13:24:04.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-13T13:24:04.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-13T13:24:04.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.200+0000 7f0a7e930640 1 Processor -- start 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.201+0000 7f0a7e930640 1 -- start start 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.201+0000 7f0a7e930640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7810b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.201+0000 7f0a7e930640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78079e70 con 0x7f0a78079a70 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.201+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7810b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.201+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7810b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35006/0 (socket says 172.21.15.43:35006) 2024-09-13T13:24:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.201+0000 7f0a77fff640 1 -- 172.21.15.43:0/631401595 learned_addr learned my addr 172.21.15.43:0/631401595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.201+0000 7f0a77fff640 1 -- 172.21.15.43:0/631401595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7810bb00 con 0x7f0a78079a70 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.202+0000 7f0a77fff640 1 --2- 172.21.15.43:0/631401595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7810b5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009820 tx=0x7f0a6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26631075ca0f8d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.202+0000 7f0a76ffd640 1 -- 172.21.15.43:0/631401595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a68037070 con 0x7f0a78079a70 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.202+0000 7f0a76ffd640 1 -- 172.21.15.43:0/631401595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0a68032890 con 0x7f0a78079a70 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.202+0000 7f0a76ffd640 1 -- 172.21.15.43:0/631401595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a68030550 con 0x7f0a78079a70 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.203+0000 7f0a7e930640 1 -- 172.21.15.43:0/631401595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 msgr2=0x7f0a7810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.203+0000 7f0a7e930640 1 --2- 172.21.15.43:0/631401595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7810b5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009820 tx=0x7f0a6802a270 comp rx=0 tx=0).stop 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.203+0000 7f0a7e930640 1 -- 172.21.15.43:0/631401595 shutdown_connections 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.203+0000 7f0a7e930640 1 --2- 172.21.15.43:0/631401595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7810b5c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.203+0000 7f0a7e930640 1 -- 172.21.15.43:0/631401595 >> 172.21.15.43:0/631401595 conn(0x7f0a78100df0 msgr2=0x7f0a78103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.203+0000 7f0a7e930640 1 -- 172.21.15.43:0/631401595 shutdown_connections 2024-09-13T13:24:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.203+0000 7f0a7e930640 1 -- 172.21.15.43:0/631401595 wait complete. 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.204+0000 7f0a7e930640 1 Processor -- start 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.204+0000 7f0a7e930640 1 -- start start 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.204+0000 7f0a7e930640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.204+0000 7f0a7e930640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7810cb50 con 0x7f0a78079a70 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.204+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.204+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35014/0 (socket says 172.21.15.43:35014) 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.204+0000 7f0a77fff640 1 -- 172.21.15.43:0/2390626550 learned_addr learned my addr 172.21.15.43:0/2390626550 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.205+0000 7f0a77fff640 1 -- 172.21.15.43:0/2390626550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7819cd90 con 0x7f0a78079a70 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.205+0000 7f0a77fff640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7819c830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a680097f0 tx=0x7f0a68030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.205+0000 7f0a757fa640 1 -- 172.21.15.43:0/2390626550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a68037040 con 0x7f0a78079a70 2024-09-13T13:24:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.205+0000 7f0a757fa640 1 -- 172.21.15.43:0/2390626550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0a68040070 con 0x7f0a78079a70 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.205+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7819d020 con 0x7f0a78079a70 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.205+0000 7f0a757fa640 1 -- 172.21.15.43:0/2390626550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a6803bd00 con 0x7f0a78079a70 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.205+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7819d9e0 con 0x7f0a78079a70 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.206+0000 7f0a757fa640 1 -- 172.21.15.43:0/2390626550 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) ==== 51778+0+0 (secure 0 0 0) 0x7f0a68045460 con 0x7f0a78079a70 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.206+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7810b5c0 con 0x7f0a78079a70 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.206+0000 7f0a757fa640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 0x7f0a60041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.207+0000 7f0a757fa640 1 -- 172.21.15.43:0/2390626550 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a680741f0 con 0x7f0a78079a70 2024-09-13T13:24:04.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.207+0000 7f0a777fe640 1 -- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 msgr2=0x7f0a60041bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/85481396 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.207+0000 7f0a777fe640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 0x7f0a60041bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.215+0000 7f0a757fa640 1 -- 172.21.15.43:0/2390626550 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f0a680443c0 con 0x7f0a78079a70 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.408+0000 7f0a777fe640 1 -- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 msgr2=0x7f0a60041bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/85481396 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.408+0000 7f0a777fe640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 0x7f0a60041bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.452+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0a7819e120 con 0x7f0a78079a70 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.453+0000 7f0a757fa640 1 -- 172.21.15.43:0/2390626550 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v10) ==== 56+0+89 (secure 0 0 0) 0x7f0a68030c60 con 0x7f0a78079a70 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 msgr2=0x7f0a60041bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 0x7f0a60041bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 msgr2=0x7f0a7819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7819c830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a680097f0 tx=0x7f0a68030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 shutdown_connections 2024-09-13T13:24:04.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f0a6003f700 0x7f0a60041bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:04.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 --2- 172.21.15.43:0/2390626550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a78079a70 0x7f0a7819c830 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:04.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 >> 172.21.15.43:0/2390626550 conn(0x7f0a78100df0 msgr2=0x7f0a78101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:04.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 shutdown_connections 2024-09-13T13:24:04.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:03.456+0000 7f0a7e930640 1 -- 172.21.15.43:0/2390626550 wait complete. 2024-09-13T13:24:04.022 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-13T13:24:04.022 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 10... 2024-09-13T13:24:04.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2390626550' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-13T13:24:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: Active manager daemon y restarted 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: Activating manager daemon y 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: osdmap e3: 0 total, 0 up, 0 in 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: mgrmap e11: y(active, starting, since 0.00364608s) 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T13:24:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:07 smithi043 ceph-mon[30106]: Manager daemon y is now available 2024-09-13T13:24:08.419 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:08.419 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:24:08.419 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:24:08.419 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-mon[30106]: mgrmap e12: y(active, since 1.00728s) 2024-09-13T13:24:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-13T13:24:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-13T13:24:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T13:24:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-13T13:24:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.465+0000 7f6c83d41640 1 Processor -- start 2024-09-13T13:24:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.465+0000 7f6c83d41640 1 -- start start 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.465+0000 7f6c83d41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c109260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.465+0000 7f6c83d41640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c1097a0 con 0x7f6c7c079ae0 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.465+0000 7f6c81add640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.465+0000 7f6c81add640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35018/0 (socket says 172.21.15.43:35018) 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.465+0000 7f6c81add640 1 -- 172.21.15.43:0/554980595 learned_addr learned my addr 172.21.15.43:0/554980595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.466+0000 7f6c81add640 1 -- 172.21.15.43:0/554980595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c109920 con 0x7f6c7c079ae0 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.467+0000 7f6c81add640 1 --2- 172.21.15.43:0/554980595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c109260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009820 tx=0x7f6c7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e804082021567960 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.467+0000 7f6c80adb640 1 -- 172.21.15.43:0/554980595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c78037070 con 0x7f6c7c079ae0 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.467+0000 7f6c80adb640 1 -- 172.21.15.43:0/554980595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6c78032890 con 0x7f6c7c079ae0 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.467+0000 7f6c80adb640 1 -- 172.21.15.43:0/554980595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c78030550 con 0x7f6c7c079ae0 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 -- 172.21.15.43:0/554980595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 msgr2=0x7f6c7c109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 --2- 172.21.15.43:0/554980595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c109260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009820 tx=0x7f6c7802a270 comp rx=0 tx=0).stop 2024-09-13T13:24:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 -- 172.21.15.43:0/554980595 shutdown_connections 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 --2- 172.21.15.43:0/554980595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c109260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 -- 172.21.15.43:0/554980595 >> 172.21.15.43:0/554980595 conn(0x7f6c7c100df0 msgr2=0x7f6c7c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 -- 172.21.15.43:0/554980595 shutdown_connections 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 -- 172.21.15.43:0/554980595 wait complete. 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 Processor -- start 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 -- start start 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.468+0000 7f6c83d41640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10a910 con 0x7f6c7c079ae0 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c81add640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c81add640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35030/0 (socket says 172.21.15.43:35030) 2024-09-13T13:24:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c81add640 1 -- 172.21.15.43:0/1162563735 learned_addr learned my addr 172.21.15.43:0/1162563735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c81add640 1 -- 172.21.15.43:0/1162563735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c19cd90 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c81add640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c19c830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c780097f0 tx=0x7f6c78030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c78037040 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6c78040070 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c19d020 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7803bd00 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.469+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c19d9e0 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.470+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) ==== 51778+0+0 (secure 0 0 0) 0x7f6c7803b430 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.471+0000 7f6c72ffd640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 0x7f6c68041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.471+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 --> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6c68042270 con 0x7f6c6803f6a0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.471+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6c780741f0 con 0x7f6c7c079ae0 2024-09-13T13:24:08.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.471+0000 7f6c812dc640 1 -- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 msgr2=0x7f6c68041b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/85481396 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.471+0000 7f6c812dc640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 0x7f6c68041b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.672+0000 7f6c812dc640 1 -- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 msgr2=0x7f6c68041b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/85481396 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:04.672+0000 7f6c812dc640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 0x7f6c68041b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:05.073+0000 7f6c812dc640 1 -- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 msgr2=0x7f6c68041b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/85481396 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:05.073+0000 7f6c812dc640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 0x7f6c68041b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:05.874+0000 7f6c812dc640 1 -- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 msgr2=0x7f6c68041b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/85481396 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:05.874+0000 7f6c812dc640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 0x7f6c68041b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:07.151+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 11) ==== 51545+0+0 (secure 0 0 0) 0x7f6c78072ca0 con 0x7f6c7c079ae0 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:07.151+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 msgr2=0x7f6c68041b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:07.151+0000 7f6c72ffd640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 0x7f6c68041b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.153+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 12) ==== 51672+0+0 (secure 0 0 0) 0x7f6c78045d10 con 0x7f6c7c079ae0 2024-09-13T13:24:08.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.153+0000 7f6c72ffd640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6c68042fe0 0x7f6c680453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.153+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6c78001d30 con 0x7f6c68042fe0 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.154+0000 7f6c812dc640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6c68042fe0 0x7f6c680453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.155+0000 7f6c812dc640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6c68042fe0 0x7f6c680453d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c003980 tx=0x7f6c6c006e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.156+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6c78001d30 con 0x7f6c68042fe0 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.163+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6c7c107250 con 0x7f6c68042fe0 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c72ffd640 1 -- 172.21.15.43:0/1162563735 <== mgr.14152 v2:172.21.15.43:6800/1020071518 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6c7c107250 con 0x7f6c68042fe0 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6c68042fe0 msgr2=0x7f6c680453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6c68042fe0 0x7f6c680453d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c003980 tx=0x7f6c6c006e10 comp rx=0 tx=0).stop 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 msgr2=0x7f6c7c19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c19c830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c780097f0 tx=0x7f6c78030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 shutdown_connections 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6c68042fe0 0x7f6c680453d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:08.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:6800/85481396,v1:172.21.15.43:6801/85481396] conn(0x7f6c6803f6a0 0x7f6c68041b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:08.677 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 --2- 172.21.15.43:0/1162563735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7c079ae0 0x7f6c7c19c830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:08.677 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 >> 172.21.15.43:0/1162563735 conn(0x7f6c7c100df0 msgr2=0x7f6c7c1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:08.677 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 shutdown_connections 2024-09-13T13:24:08.677 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:08.164+0000 7f6c83d41640 1 -- 172.21.15.43:0/1162563735 wait complete. 2024-09-13T13:24:08.677 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 10 is available 2024-09-13T13:24:08.677 INFO:teuthology.orchestra.run.smithi043.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-13T13:24:09.203 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:24:08.818+0000 7f1519f01640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:13:24:08] ENGINE Error in HTTPServer.serve 2024-09-13T13:24:09.204 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: Traceback (most recent call last): 2024-09-13T13:24:09.204 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:24:09.204 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._connections.run(self.expiration_interval) 2024-09-13T13:24:09.204 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:24:09.204 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._run(expiration_interval) 2024-09-13T13:24:09.204 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:24:09.204 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:24:09.205 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:24:09.205 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:24:09.205 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:24:09.205 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: s = self.context.wrap_socket( 2024-09-13T13:24:09.205 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:24:09.205 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: return self.sslsocket_class._create( 2024-09-13T13:24:09.205 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:24:09.206 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self.do_handshake() 2024-09-13T13:24:09.206 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:24:09.206 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._sslobj.do_handshake() 2024-09-13T13:24:09.206 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:24:09.206 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:24:08 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:24:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:09 smithi043 ceph-mon[30106]: [13/Sep/2024:13:24:08] ENGINE Bus STARTING 2024-09-13T13:24:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:09 smithi043 ceph-mon[30106]: [13/Sep/2024:13:24:08] ENGINE Serving on https://172.21.15.43:7150 2024-09-13T13:24:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:09 smithi043 ceph-mon[30106]: [13/Sep/2024:13:24:08] ENGINE Error in HTTPServer.serve 2024-09-13T13:24:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-13T13:24:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:24:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-13T13:24:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:24:09.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-13T13:24:09.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:24:09.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-13T13:24:09.494 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:24:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:10 smithi043 ceph-mon[30106]: [13/Sep/2024:13:24:08] ENGINE Serving on http://172.21.15.43:8765 2024-09-13T13:24:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:10 smithi043 ceph-mon[30106]: [13/Sep/2024:13:24:08] ENGINE Bus STARTED 2024-09-13T13:24:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:10 smithi043 ceph-mon[30106]: mgrmap e13: y(active, since 2s) 2024-09-13T13:24:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:10 smithi043 ceph-mon[30106]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:10 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:11.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:11 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUCkZ+fWAThLAMzowU3uRxFL96uWQwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMzEzMjQxMFoXDTM0\nMDkxNDEzMjQxMFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjQzMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAqzkoQvZ62ozSKfeaexCf/2CHi1I01W74mcMq\nmHB5YcBMMZW5mNbPps4lMKfwyql4PO5TZuOHVxyS1NaJCMDFSbKWJzs7Ez5Ll0Dj\n60qGzyMyPw2bnHR0s8dyMuHanNEikRNtCMIE3TTLOHgbtO0mutFeeMFt9ayqpEWy\njSN2rPne5ly924s6foSIyzSqz9KXw+hltUW+kQtApUxRPV961t0j6rF5iCklcUdz\n3edJIjsylwB181RJHzvPO7/CjNy0bAiEkHT1sMjJr8aoC4rlKpM+SlIT/E8cBweo\nmUNU0BM1fV/6iFJDJDlt5L/VsC6J+MHc39OpwWcjgujEReqJDQVCcElBApzw1b+U\nOYdk3M/t3t+bfR7rUke2C3p0mK/vgOZBtXqm2ZPE/+S6WZe81HBQ+DBZJtv8Nm/R\nrKsCshAeSci2/8fBvgLlmIuhOWUvhAeQDmcG6dj1TkF57qJd/fC10SKzunqCl0i/\nlqhQ4laU811GR/OvkvCTQ7NzQnypWPG+0uu+QBY4vbary+gFhvVS06qDL2UkqKbi\n6DAFh5JYJr8qFC2ZF4m2H9wD+Z5/RD61jZDc02t9r6bsnrjZkEkQJcPw43cYg5KU\nnyjLAn/NwPvistJG93brmRgCVGWstxTYTwqBCp3qSj+9Lh4J6E9nk63eahzZcu6d\nEwsNs9kCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDQzghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPKzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAp\nxUP8a04o+ZzKE5IoYlknWb63Ew97V2P2GRKwNhPSExNOTE7yR3i4xnW+AyGesRaN\nIb/+I7nCoW046V+xKJWKqmkyH4DCmIQhhFsXdkTknem0iZOVdVvLgLXcgX3Aeyf/\n7AjbMUtBQZ+rdFjlyAg8lhRJd6E3WXhpQi0b4x/32HxKDzqHZlLQWwZ54hlN8+gW\nOx+9PCX21hBMRKeoxmzylNz7dHXYsON87D3Cp6Qc/f/q3GyeIBAb9MtK46UDsNPP\npomHArkC50XHIp+/aVQLQFwOhhLSD/0mESpzkAp+IJe/Yt878GQjLKOppE/yNnry\n4qIt/+qd/k/vjAxeTlQx7SOfABWotOp21Sc3SIDYlCIxsNZL/Y6851ZR4c1BJ2/i\nED29aEWFMtWEtFa5wtOotpC9aiDkTPYQYmg5xBPP60NwyS6yJ3eGAemOIXgHtArE\ns2iMOSAWtT0UGTdoXswvu0vh8lXLQgOHIKZjJPZFownTUahzSxQT8waAmhLTuZvQ\nWk0LGtf7HMnStP0xqpojo064BoGDxy8ZaTBpz02Igj8l6u46Uh8zZCIbrwD3Z/JE\nKXuYAnMbZ76pmMzveTYb9EduNJGAebpzHrtEzo850bCLsqmPcOVUU7ZnbDp+6lAB\nr13cg6h/+1rI9oRAe2yA0Ms647huRAMkGtgXhSOxNg==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKgIBAAKCAgEAqzkoQvZ62ozSKfeaexCf/2CHi1I01W74mcMqmHB5YcBMMZW5\nmNbPps4lMKfwyql4PO5TZuOHVxyS1NaJCMDFSbKWJzs7Ez5Ll0Dj60qGzyMyPw2b\nnHR0s8dyMuHanNEikRNtCMIE3TTLOHgbtO0mutFeeMFt9ayqpEWyjSN2rPne5ly9\n24s6foSIyzSqz9KXw+hltUW+kQtApUxRPV961t0j6rF5iCklcUdz3edJIjsylwB1\n81RJHzvPO7/CjNy0bAiEkHT1sMjJr8aoC4rlKpM+SlIT/E8cBweomUNU0BM1fV/6\niFJDJDlt5L/VsC6J+MHc39OpwWcjgujEReqJDQVCcElBApzw1b+UOYdk3M/t3t+b\nfR7rUke2C3p0mK/vgOZBtXqm2ZPE/+S6WZe81HBQ+DBZJtv8Nm/RrKsCshAeSci2\n/8fBvgLlmIuhOWUvhAeQDmcG6dj1TkF57qJd/fC10SKzunqCl0i/lqhQ4laU811G\nR/OvkvCTQ7NzQnypWPG+0uu+QBY4vbary+gFhvVS06qDL2UkqKbi6DAFh5JYJr8q\nFC2ZF4m2H9wD+Z5/RD61jZDc02t9r6bsnrjZkEkQJcPw43cYg5KUnyjLAn/NwPvi\nstJG93brmRgCVGWstxTYTwqBCp3qSj+9Lh4J6E9nk63eahzZcu6dEwsNs9kCAwEA\nAQKCAgAIvB+7sCClqZgcdU83hqpC4u/HQb2u7bxXoRyOg55i8uZwLnfgvi9VuR/Y\nku4LyLLOQ5zpm3iPZHEb0937HlUc5LQYSEND62JRKF74OpMUr3JOw9OecFUhHsRK\nqDmjf7UduuSjFRLJHiUH5R4KECxw92FfE68Xtg6hLBURx4S0X62UCZh97LpiiJcE\nltPn9OcHzYFIIB4P/tz2EdCggDsfXvoIF7RgJ96LIDQJq/+PyX6pNBLQWQpp7NPY\n881pq5C2D4aDwBO7SGL8SUwlWplE9MIGv0KEapy+wbtDe6swpOo4/SkmGdw8VYRa\nWRb+UpbN+UXap/ekWtigQwj68B4RqzRr0IymgjoSehQIcv0rFjxY4Fgc0gu2PTeO\nd3rUi2SjnLiPNvhf0FazXKJkoDhZcjucUTtK8RPYVbI7IuMz1UGfi8wAbAjkEg3d\n/nBKy4+l3A0kZWXqlRwhyA/nUeu0RgNZ78IE27MjnfyCc7/KC6Zs8dIb/4cCiJ1q\nI5dxruXVU7/4L41WRMLDaUVWGkpNHTbc9s6hS1kEymVtXMoOgpT+2iDs7AFVG+l3\nwpCU+BIEk0/RSwUsDJaxG1HD4yXF9jZ/vV+eev23OlTI5jnSaVoDwMhbF5Ucj3LZ\nU8MNnDFvgzxK3bPmTRIMpkaW5CsISkBsvNWneO/YaF6/vunbKQKCAQEA5BUM6Orw\n1JgZSYJuJv8yvrh96nDhwPWVGq+eEkI+Q+Ya/S87OpJZw5zsa6KOVctWbKP7wgkW\noUNCtUzRfzrKBS4NEzF7pgRmvhAzmsT2sKf/BeulX0i2O0QQbh2hQ2LsAVU5vgRe\nD7paU7m/OQj00i+YVrE4wVhag65BnO4hcB+9O3qsumfyfQQ4BREcC592gV50OdEb\nR5q40C0xVh0sCXrG/sHN3Gop/RZsK4NpGcz8iwpdON04vCqO+D8Wc/gipXv8dLYG\nIn7Qdm25eKBdve/E2Z3Hi5moNlXOrKlBOXR/pyEkJDq49y5nnZaE1nxWgWrngRDJ\nnmbXCohRTZj3+wKCAQEAwC5u478K+izIMMKHghD4xqwhdBhlZfwMBfBZKgY/MmqQ\nvJjji4H8XSjdXPwh1lAbjSqE6rFYbq3RW65/Z7wpWPrLH4eKj5gRkY/QUe37QDXw\not+kNAcZGFqzKuEFNEc3lAg7reO9LCXb++eJhHJqHUvWcCsmp6vZ92N+3x11GNNn\noAM49qt2h9I+bgMTH+ca9GPlg7dUIT3DrUtMz2+8hpsBYciSGftwPG+kEQ6Cr40e\nnAEA22iTqKHDXilHkuf6u8pthJJrznw6lbGMYX/Kv5ajjrxGtKrZMGW/S55kwJVz\nW3SlT2xzXrskGgQqkwGQYQygIjTfCDlvg8TpH38XOwKCAQEAlohmKtvU8xsnpWKW\nPNq1rTaqtC6ocOgk7qX11+ssmTennG6t0qyA3iAV0bxJ4MoBUlawdJH3A5wWzP3U\nTvqIFv8b1g0UpK/G965NwNo6TsGyCUBLtyWQUHFNvr+DjaweQ4ryPz02r3642wV+\n9a+5lXegPadd5A3iP4Kk/2QTYaMxC66djbHJU0WTi5SNRAaM+jlS0E34iTvOumb5\nZvJ9/AuyhrV2nwDakZVY8ycaGNM34rSGbl9jcJ1OGv3YXdXpV/sK1iTEy8OS5nbo\nsHp3JEK5Uz9VUgeqA6KCc2aKC9ed4Lp41giDx3AzJskPB1is52GdRYvE5ezOBL+P\nmX1aPQKCAQEAprdhQTcY+cBhhlMrJL0+Y3Mtrj3/sMI/V+OSJi+3ITBrEQD83KhN\n2i86IZEkdXuW+xZ4qfRhR9qH9SXipoYHxIYEMfzuSfVcwE7nJDv0K1bfwxuJogzK\nS2NbbIY6coiZgVTHGOmo0SSiutu4Y2qx/Wotw67bYLdnZgItLLasP3YdviUthGdo\ncxkDSYJnU9SzgHEsyOAAHrorByl8n7NJ8+eCAmbxv5JtYjBjPty8qDge5qzEJpKJ\nDdAO4J+k1J9j1wcbjNdMqJsCY5vbsTUgGKZAivFv0QkpUp/hnGLy+lJr+VmuksD9\nMJ/eCKe2m0LPEjHWtPXyyXqx77ihl84zwwKCAQEAwqYxU/5p8Y6W4NCgZiOQD86N\nKE5giMcb71Ql3mLh3UI55j1HD2TM8jfAijBzSeg/pdxc0L0e+WM/4MPhEzuNX246\nwSSAi1mxmB0c6d4ufTqIRet3bi3THes6yO8jSHVbTMyo8UPiQ2lebLTGFTUuO3uE\nizWNf7Kgn0cFtKpMD3gVjBfDBTKKbo29kVec14NwCa/prC7UBmLy0/l+cNTO+ffo\nS9HRPGdStdlOlnsd6aRDofUL6/vCYd5Bhe3vHzTCNsyOYQDzcIo7AvuCmlGsdnV1\n0J7L3hSwAf4KSUiblmL2pju9j0uKfNF7BfS1e2Q1bQCabWWmxVsA/OvusOKx/Q==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.291+0000 7ffb6b76c640 1 Processor -- start 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.291+0000 7ffb6b76c640 1 -- start start 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.292+0000 7ffb6b76c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb64106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.292+0000 7ffb6b76c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64107000 con 0x7ffb64106610 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.292+0000 7ffb69508640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb64106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.292+0000 7ffb69508640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb64106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58972/0 (socket says 172.21.15.43:58972) 2024-09-13T13:24:11.997 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.292+0000 7ffb69508640 1 -- 172.21.15.43:0/2663109698 learned_addr learned my addr 172.21.15.43:0/2663109698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.292+0000 7ffb69508640 1 -- 172.21.15.43:0/2663109698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64107180 con 0x7ffb64106610 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.293+0000 7ffb69508640 1 --2- 172.21.15.43:0/2663109698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb64106a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009820 tx=0x7ffb5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850d706c566c173d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.293+0000 7ffb63fff640 1 -- 172.21.15.43:0/2663109698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c037070 con 0x7ffb64106610 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.293+0000 7ffb63fff640 1 -- 172.21.15.43:0/2663109698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ffb5c032890 con 0x7ffb64106610 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.293+0000 7ffb63fff640 1 -- 172.21.15.43:0/2663109698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c030550 con 0x7ffb64106610 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 -- 172.21.15.43:0/2663109698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 msgr2=0x7ffb64106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 --2- 172.21.15.43:0/2663109698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb64106a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009820 tx=0x7ffb5c02a270 comp rx=0 tx=0).stop 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 -- 172.21.15.43:0/2663109698 shutdown_connections 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 --2- 172.21.15.43:0/2663109698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb64106a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 -- 172.21.15.43:0/2663109698 >> 172.21.15.43:0/2663109698 conn(0x7ffb64101f00 msgr2=0x7ffb64104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 -- 172.21.15.43:0/2663109698 shutdown_connections 2024-09-13T13:24:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 -- 172.21.15.43:0/2663109698 wait complete. 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 Processor -- start 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.294+0000 7ffb6b76c640 1 -- start start 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb6b76c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb6419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb6b76c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6410aa90 con 0x7ffb64106610 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb69508640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb6419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb69508640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb6419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58976/0 (socket says 172.21.15.43:58976) 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb69508640 1 -- 172.21.15.43:0/4003716338 learned_addr learned my addr 172.21.15.43:0/4003716338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb69508640 1 -- 172.21.15.43:0/4003716338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6419ab20 con 0x7ffb64106610 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb69508640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb6419a5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0097f0 tx=0x7ffb5c030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb627fc640 1 -- 172.21.15.43:0/4003716338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c037040 con 0x7ffb64106610 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6419adb0 con 0x7ffb64106610 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb627fc640 1 -- 172.21.15.43:0/4003716338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ffb5c040070 con 0x7ffb64106610 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb627fc640 1 -- 172.21.15.43:0/4003716338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c03bd40 con 0x7ffb64106610 2024-09-13T13:24:11.999 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.295+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6419b770 con 0x7ffb64106610 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.296+0000 7ffb627fc640 1 -- 172.21.15.43:0/4003716338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 51778+0+0 (secure 0 0 0) 0x7ffb5c045460 con 0x7ffb64106610 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.296+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6410ac10 con 0x7ffb64106610 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.297+0000 7ffb627fc640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ffb4c03f6a0 0x7ffb4c041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.297+0000 7ffb627fc640 1 -- 172.21.15.43:0/4003716338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffb5c074320 con 0x7ffb64106610 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.297+0000 7ffb68d07640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ffb4c03f6a0 0x7ffb4c041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.297+0000 7ffb68d07640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ffb4c03f6a0 0x7ffb4c041b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800ad30 tx=0x7ffb580093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.302+0000 7ffb627fc640 1 -- 172.21.15.43:0/4003716338 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ffb5c036170 con 0x7ffb64106610 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:09.496+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}) -- 0x7ffb64103210 con 0x7ffb4c03f6a0 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.591+0000 7ffb627fc640 1 -- 172.21.15.43:0/4003716338 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+5140 (secure 0 0 0) 0x7ffb64103210 con 0x7ffb4c03f6a0 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ffb4c03f6a0 msgr2=0x7ffb4c041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ffb4c03f6a0 0x7ffb4c041b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800ad30 tx=0x7ffb580093f0 comp rx=0 tx=0).stop 2024-09-13T13:24:12.000 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 msgr2=0x7ffb6419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb6419a5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0097f0 tx=0x7ffb5c030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 shutdown_connections 2024-09-13T13:24:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ffb4c03f6a0 0x7ffb4c041b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 --2- 172.21.15.43:0/4003716338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb64106610 0x7ffb6419a5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 >> 172.21.15.43:0/4003716338 conn(0x7ffb64101f00 msgr2=0x7ffb64102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 shutdown_connections 2024-09-13T13:24:12.001 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:10.593+0000 7ffb6b76c640 1 -- 172.21.15.43:0/4003716338 wait complete. 2024-09-13T13:24:13.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-13T13:24:13.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.436+0000 7f6594423640 1 Processor -- start 2024-09-13T13:24:13.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.436+0000 7f6594423640 1 -- start start 2024-09-13T13:24:13.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.437+0000 7f6594423640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c1058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.437+0000 7f6594423640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f658c105eb0 con 0x7f658c1054c0 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.437+0000 7f65921bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c1058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.437+0000 7f65921bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c1058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58986/0 (socket says 172.21.15.43:58986) 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.437+0000 7f65921bf640 1 -- 172.21.15.43:0/4065027799 learned_addr learned my addr 172.21.15.43:0/4065027799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.437+0000 7f65921bf640 1 -- 172.21.15.43:0/4065027799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c106030 con 0x7f658c1054c0 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.438+0000 7f65921bf640 1 --2- 172.21.15.43:0/4065027799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c1058c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6584009820 tx=0x7f658402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ea51a6ff85bcfb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.438+0000 7f65911bd640 1 -- 172.21.15.43:0/4065027799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6584037070 con 0x7f658c1054c0 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.438+0000 7f65911bd640 1 -- 172.21.15.43:0/4065027799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6584032890 con 0x7f658c1054c0 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.438+0000 7f65911bd640 1 -- 172.21.15.43:0/4065027799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6584030550 con 0x7f658c1054c0 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 -- 172.21.15.43:0/4065027799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 msgr2=0x7f658c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 --2- 172.21.15.43:0/4065027799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c1058c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6584009820 tx=0x7f658402a270 comp rx=0 tx=0).stop 2024-09-13T13:24:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 -- 172.21.15.43:0/4065027799 shutdown_connections 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 --2- 172.21.15.43:0/4065027799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c1058c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 -- 172.21.15.43:0/4065027799 >> 172.21.15.43:0/4065027799 conn(0x7f658c100df0 msgr2=0x7f658c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 -- 172.21.15.43:0/4065027799 shutdown_connections 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 -- 172.21.15.43:0/4065027799 wait complete. 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 Processor -- start 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.439+0000 7f6594423640 1 -- start start 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f6594423640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f6594423640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f658c109940 con 0x7f658c1054c0 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f65921bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f65921bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58990/0 (socket says 172.21.15.43:58990) 2024-09-13T13:24:13.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f65921bf640 1 -- 172.21.15.43:0/112191199 learned_addr learned my addr 172.21.15.43:0/112191199 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f65921bf640 1 -- 172.21.15.43:0/112191199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c19aba0 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f65921bf640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c19a640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65840097f0 tx=0x7f6584030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f658b7fe640 1 -- 172.21.15.43:0/112191199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6584037040 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f658b7fe640 1 -- 172.21.15.43:0/112191199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6584040070 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658c19ae30 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f658b7fe640 1 -- 172.21.15.43:0/112191199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f658403bd00 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.440+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658c19b7f0 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.441+0000 7f658b7fe640 1 -- 172.21.15.43:0/112191199 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 51778+0+0 (secure 0 0 0) 0x7f6584045460 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.441+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658c109ac0 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.442+0000 7f658b7fe640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f657403f700 0x7f6574041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.442+0000 7f658b7fe640 1 -- 172.21.15.43:0/112191199 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6584074290 con 0x7f658c1054c0 2024-09-13T13:24:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.442+0000 7f65919be640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f657403f700 0x7f6574041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.442+0000 7f65919be640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f657403f700 0x7f6574041bc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65800098c0 tx=0x7f6580006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.446+0000 7f658b7fe640 1 -- 172.21.15.43:0/112191199 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f6584030c60 con 0x7f658c1054c0 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.648+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}) -- 0x7f658c079680 con 0x7f657403f700 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.651+0000 7f658b7fe640 1 -- 172.21.15.43:0/112191199 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f658c079680 con 0x7f657403f700 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f657403f700 msgr2=0x7f6574041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f657403f700 0x7f6574041bc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65800098c0 tx=0x7f6580006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 msgr2=0x7f658c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c19a640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65840097f0 tx=0x7f6584030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 shutdown_connections 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f657403f700 0x7f6574041bc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 --2- 172.21.15.43:0/112191199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f658c1054c0 0x7f658c19a640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 >> 172.21.15.43:0/112191199 conn(0x7f658c100df0 msgr2=0x7f658c078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:13.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 shutdown_connections 2024-09-13T13:24:13.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:12.653+0000 7f6594423640 1 -- 172.21.15.43:0/112191199 wait complete. 2024-09-13T13:24:13.872 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:13 smithi043 ceph-mon[30106]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:13.872 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:13 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:14.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-13T13:24:14.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.660+0000 7f86694b8640 1 Processor -- start 2024-09-13T13:24:14.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.660+0000 7f86694b8640 1 -- start start 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.660+0000 7f86694b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f86641058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.660+0000 7f86694b8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8664105eb0 con 0x7f86641054c0 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.661+0000 7f8662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f86641058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.661+0000 7f8662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f86641058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59006/0 (socket says 172.21.15.43:59006) 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.661+0000 7f8662ffd640 1 -- 172.21.15.43:0/837837520 learned_addr learned my addr 172.21.15.43:0/837837520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.661+0000 7f8662ffd640 1 -- 172.21.15.43:0/837837520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8664106030 con 0x7f86641054c0 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.662+0000 7f8662ffd640 1 --2- 172.21.15.43:0/837837520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f86641058c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8654009820 tx=0x7f865402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f66228af74c6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.662+0000 7f86627fc640 1 -- 172.21.15.43:0/837837520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8654037070 con 0x7f86641054c0 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.662+0000 7f86627fc640 1 -- 172.21.15.43:0/837837520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f8654032890 con 0x7f86641054c0 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.662+0000 7f86627fc640 1 -- 172.21.15.43:0/837837520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8654030550 con 0x7f86641054c0 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.662+0000 7f86694b8640 1 -- 172.21.15.43:0/837837520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 msgr2=0x7f86641058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.662+0000 7f86694b8640 1 --2- 172.21.15.43:0/837837520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f86641058c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8654009820 tx=0x7f865402a270 comp rx=0 tx=0).stop 2024-09-13T13:24:14.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.663+0000 7f86694b8640 1 -- 172.21.15.43:0/837837520 shutdown_connections 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.663+0000 7f86694b8640 1 --2- 172.21.15.43:0/837837520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f86641058c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.663+0000 7f86694b8640 1 -- 172.21.15.43:0/837837520 >> 172.21.15.43:0/837837520 conn(0x7f8664100df0 msgr2=0x7f8664103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.663+0000 7f86694b8640 1 -- 172.21.15.43:0/837837520 shutdown_connections 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.663+0000 7f86694b8640 1 -- 172.21.15.43:0/837837520 wait complete. 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.663+0000 7f86694b8640 1 Processor -- start 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.663+0000 7f86694b8640 1 -- start start 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f86694b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f866419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f86694b8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8664109940 con 0x7f86641054c0 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f866419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f866419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59022/0 (socket says 172.21.15.43:59022) 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8662ffd640 1 -- 172.21.15.43:0/1054591152 learned_addr learned my addr 172.21.15.43:0/1054591152 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8662ffd640 1 -- 172.21.15.43:0/1054591152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866419aba0 con 0x7f86641054c0 2024-09-13T13:24:14.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8662ffd640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f866419a640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86540097f0 tx=0x7f8654030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8660ff9640 1 -- 172.21.15.43:0/1054591152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8654037040 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8660ff9640 1 -- 172.21.15.43:0/1054591152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f8654040070 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866419ae30 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f8660ff9640 1 -- 172.21.15.43:0/1054591152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f865403bd00 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.664+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866419b7f0 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.665+0000 7f8660ff9640 1 -- 172.21.15.43:0/1054591152 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 51778+0+0 (secure 0 0 0) 0x7f865403b430 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.665+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8664109ac0 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.666+0000 7f8660ff9640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f865003f700 0x7f8650041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.666+0000 7f8660ff9640 1 -- 172.21.15.43:0/1054591152 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8654074300 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.666+0000 7f865bfff640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f865003f700 0x7f8650041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.666+0000 7f865bfff640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f865003f700 0x7f8650041bc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f864c0098c0 tx=0x7f864c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.670+0000 7f8660ff9640 1 -- 172.21.15.43:0/1054591152 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f86540313a0 con 0x7f86641054c0 2024-09-13T13:24:14.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.870+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}) -- 0x7f86641058c0 con 0x7f865003f700 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.874+0000 7f8660ff9640 1 -- 172.21.15.43:0/1054591152 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f86641058c0 con 0x7f865003f700 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f865003f700 msgr2=0x7f8650041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f865003f700 0x7f8650041bc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f864c0098c0 tx=0x7f864c006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 msgr2=0x7f866419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f866419a640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86540097f0 tx=0x7f8654030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 shutdown_connections 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f865003f700 0x7f8650041bc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 --2- 172.21.15.43:0/1054591152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86641054c0 0x7f866419a640 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 >> 172.21.15.43:0/1054591152 conn(0x7f8664100df0 msgr2=0x7f8664078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 shutdown_connections 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:13.877+0000 7f86694b8640 1 -- 172.21.15.43:0/1054591152 wait complete. 2024-09-13T13:24:14.421 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-09-13T13:24:15.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:14 smithi043 ceph-mon[30106]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:15.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:14 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:15.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$wGvdwPrii2R/URfNgkvXOeCtFhLdUx2iyS1xpIBIt5WKKXzYhBOyu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726233855, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.836+0000 7f97c6a4a640 1 Processor -- start 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.837+0000 7f97c6a4a640 1 -- start start 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.837+0000 7f97c6a4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c01058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.837+0000 7f97c6a4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0105eb0 con 0x7f97c01054c0 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.837+0000 7f97bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c01058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.837+0000 7f97bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c01058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59036/0 (socket says 172.21.15.43:59036) 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.837+0000 7f97bffff640 1 -- 172.21.15.43:0/295674761 learned_addr learned my addr 172.21.15.43:0/295674761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.837+0000 7f97bffff640 1 -- 172.21.15.43:0/295674761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c0106030 con 0x7f97c01054c0 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.838+0000 7f97bffff640 1 --2- 172.21.15.43:0/295674761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c01058c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009820 tx=0x7f97b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d413579f33794fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.838+0000 7f97beffd640 1 -- 172.21.15.43:0/295674761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b0037070 con 0x7f97c01054c0 2024-09-13T13:24:15.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.838+0000 7f97beffd640 1 -- 172.21.15.43:0/295674761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f97b0032890 con 0x7f97c01054c0 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.838+0000 7f97beffd640 1 -- 172.21.15.43:0/295674761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b0030550 con 0x7f97c01054c0 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.839+0000 7f97c6a4a640 1 -- 172.21.15.43:0/295674761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 msgr2=0x7f97c01058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.839+0000 7f97c6a4a640 1 --2- 172.21.15.43:0/295674761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c01058c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009820 tx=0x7f97b002a270 comp rx=0 tx=0).stop 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.839+0000 7f97c6a4a640 1 -- 172.21.15.43:0/295674761 shutdown_connections 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.839+0000 7f97c6a4a640 1 --2- 172.21.15.43:0/295674761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c01058c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.839+0000 7f97c6a4a640 1 -- 172.21.15.43:0/295674761 >> 172.21.15.43:0/295674761 conn(0x7f97c0100df0 msgr2=0x7f97c0103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.839+0000 7f97c6a4a640 1 -- 172.21.15.43:0/295674761 shutdown_connections 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.839+0000 7f97c6a4a640 1 -- 172.21.15.43:0/295674761 wait complete. 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97c6a4a640 1 Processor -- start 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97c6a4a640 1 -- start start 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97c6a4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97c6a4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0109940 con 0x7f97c01054c0 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59046/0 (socket says 172.21.15.43:59046) 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97bffff640 1 -- 172.21.15.43:0/2684132398 learned_addr learned my addr 172.21.15.43:0/2684132398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97bffff640 1 -- 172.21.15.43:0/2684132398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c019ab50 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.840+0000 7f97bffff640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c019a5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97b00097f0 tx=0x7f97b0030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.841+0000 7f97bd7fa640 1 -- 172.21.15.43:0/2684132398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b0037040 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.841+0000 7f97bd7fa640 1 -- 172.21.15.43:0/2684132398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f97b0040070 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.841+0000 7f97bd7fa640 1 -- 172.21.15.43:0/2684132398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b003bd00 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.841+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c019ade0 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.841+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c019b7a0 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.842+0000 7f97bd7fa640 1 -- 172.21.15.43:0/2684132398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 51778+0+0 (secure 0 0 0) 0x7f97b003b430 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.842+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c0109ac0 con 0x7f97c01054c0 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.842+0000 7f97bd7fa640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f97a803f6a0 0x7f97a8041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.842+0000 7f97bd7fa640 1 -- 172.21.15.43:0/2684132398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f97b00742d0 con 0x7f97c01054c0 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.842+0000 7f97bf7fe640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f97a803f6a0 0x7f97a8041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.842+0000 7f97bf7fe640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f97a803f6a0 0x7f97a8041b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f97b40098c0 tx=0x7f97b4006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:14.847+0000 7f97bd7fa640 1 -- 172.21.15.43:0/2684132398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f97b00313a0 con 0x7f97c01054c0 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.045+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f97c01058c0 con 0x7f97a803f6a0 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.294+0000 7f97bd7fa640 1 -- 172.21.15.43:0/2684132398 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f97c01058c0 con 0x7f97a803f6a0 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f97a803f6a0 msgr2=0x7f97a8041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f97a803f6a0 0x7f97a8041b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f97b40098c0 tx=0x7f97b4006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 msgr2=0x7f97c019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c019a5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97b00097f0 tx=0x7f97b0030ad0 comp rx=0 tx=0).stop 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 shutdown_connections 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f97a803f6a0 0x7f97a8041b60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 --2- 172.21.15.43:0/2684132398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c01054c0 0x7f97c019a5f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:15.885 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 >> 172.21.15.43:0/2684132398 conn(0x7f97c0100df0 msgr2=0x7f97c0078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:15.886 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 shutdown_connections 2024-09-13T13:24:15.886 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:15.296+0000 7f97c6a4a640 1 -- 172.21.15.43:0/2684132398 wait complete. 2024-09-13T13:24:15.886 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-09-13T13:24:16.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:16 smithi043 ceph-mon[30106]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:16.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:16 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.329+0000 7fb62d7a2640 1 Processor -- start 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.329+0000 7fb62d7a2640 1 -- start start 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.329+0000 7fb62d7a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.329+0000 7fb62d7a2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb628105eb0 con 0x7fb6281054c0 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.330+0000 7fb626ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.330+0000 7fb626ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59048/0 (socket says 172.21.15.43:59048) 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.330+0000 7fb626ffd640 1 -- 172.21.15.43:0/1980242454 learned_addr learned my addr 172.21.15.43:0/1980242454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.330+0000 7fb626ffd640 1 -- 172.21.15.43:0/1980242454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb628106030 con 0x7fb6281054c0 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.331+0000 7fb626ffd640 1 --2- 172.21.15.43:0/1980242454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281058c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb620009820 tx=0x7fb62002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bc4cd431c11149 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.331+0000 7fb625ffb640 1 -- 172.21.15.43:0/1980242454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb620037070 con 0x7fb6281054c0 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.331+0000 7fb625ffb640 1 -- 172.21.15.43:0/1980242454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb620032890 con 0x7fb6281054c0 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.331+0000 7fb625ffb640 1 -- 172.21.15.43:0/1980242454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb620030550 con 0x7fb6281054c0 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.331+0000 7fb62d7a2640 1 -- 172.21.15.43:0/1980242454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 msgr2=0x7fb6281058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.331+0000 7fb62d7a2640 1 --2- 172.21.15.43:0/1980242454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281058c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb620009820 tx=0x7fb62002a270 comp rx=0 tx=0).stop 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.331+0000 7fb62d7a2640 1 -- 172.21.15.43:0/1980242454 shutdown_connections 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.332+0000 7fb62d7a2640 1 --2- 172.21.15.43:0/1980242454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281058c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.332+0000 7fb62d7a2640 1 -- 172.21.15.43:0/1980242454 >> 172.21.15.43:0/1980242454 conn(0x7fb628100df0 msgr2=0x7fb628103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.332+0000 7fb62d7a2640 1 -- 172.21.15.43:0/1980242454 shutdown_connections 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.332+0000 7fb62d7a2640 1 -- 172.21.15.43:0/1980242454 wait complete. 2024-09-13T13:24:17.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.332+0000 7fb62d7a2640 1 Processor -- start 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.332+0000 7fb62d7a2640 1 -- start start 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb62d7a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281903e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb62d7a2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb628109940 con 0x7fb6281054c0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb626ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb626ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281903e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59060/0 (socket says 172.21.15.43:59060) 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb626ffd640 1 -- 172.21.15.43:0/2603045039 learned_addr learned my addr 172.21.15.43:0/2603045039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb626ffd640 1 -- 172.21.15.43:0/2603045039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb628190940 con 0x7fb6281054c0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb626ffd640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281903e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb620032860 tx=0x7fb620030a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb61ffff640 1 -- 172.21.15.43:0/2603045039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb620037040 con 0x7fb6281054c0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb61ffff640 1 -- 172.21.15.43:0/2603045039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb620040070 con 0x7fb6281054c0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62818ed30 con 0x7fb6281054c0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb61ffff640 1 -- 172.21.15.43:0/2603045039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb62003bcc0 con 0x7fb6281054c0 2024-09-13T13:24:17.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.333+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62818f2b0 con 0x7fb6281054c0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.334+0000 7fb61ffff640 1 -- 172.21.15.43:0/2603045039 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 51778+0+0 (secure 0 0 0) 0x7fb62003b430 con 0x7fb6281054c0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.334+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb628109ac0 con 0x7fb6281054c0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.335+0000 7fb61ffff640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fb61403f700 0x7fb614041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.335+0000 7fb61ffff640 1 -- 172.21.15.43:0/2603045039 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb620074290 con 0x7fb6281054c0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.335+0000 7fb6267fc640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fb61403f700 0x7fb614041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.335+0000 7fb6267fc640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fb61403f700 0x7fb614041bc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098c0 tx=0x7fb610006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.339+0000 7fb61ffff640 1 -- 172.21.15.43:0/2603045039 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb6200313a0 con 0x7fb6281054c0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.524+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb6281058c0 con 0x7fb6281054c0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.525+0000 7fb61ffff640 1 -- 172.21.15.43:0/2603045039 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb620030d50 con 0x7fb6281054c0 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fb61403f700 msgr2=0x7fb614041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fb61403f700 0x7fb614041bc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098c0 tx=0x7fb610006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 msgr2=0x7fb6281903e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:17.090 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281903e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb620032860 tx=0x7fb620030a80 comp rx=0 tx=0).stop 2024-09-13T13:24:17.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 shutdown_connections 2024-09-13T13:24:17.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fb61403f700 0x7fb614041bc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:17.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 --2- 172.21.15.43:0/2603045039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6281054c0 0x7fb6281903e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:17.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 >> 172.21.15.43:0/2603045039 conn(0x7fb628100df0 msgr2=0x7fb628078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:17.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 shutdown_connections 2024-09-13T13:24:17.091 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:16.528+0000 7fb62d7a2640 1 -- 172.21.15.43:0/2603045039 wait complete. 2024-09-13T13:24:17.095 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T13:24:17.095 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-13T13:24:17.103 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T13:24:17.103 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-13T13:24:17.103 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-13T13:24:17.104 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-13T13:24:17.105 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-09-13T13:24:17.106 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:17.106 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-09-13T13:24:17.106 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-09-13T13:24:17.106 INFO:teuthology.orchestra.run.smithi043.stdout: Password: 678nb1lvho 2024-09-13T13:24:17.106 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:17.106 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config directory 2024-09-13T13:24:17.349 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2603045039' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.566+0000 7fa04d869640 1 Processor -- start 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.566+0000 7fa04d869640 1 -- start start 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.566+0000 7fa04d869640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa048106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.566+0000 7fa04d869640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa048107000 con 0x7fa048106610 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.566+0000 7fa046ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa048106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.566+0000 7fa046ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa048106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43096/0 (socket says 172.21.15.43:43096) 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.566+0000 7fa046ffd640 1 -- 172.21.15.43:0/3076773053 learned_addr learned my addr 172.21.15.43:0/3076773053 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.567+0000 7fa046ffd640 1 -- 172.21.15.43:0/3076773053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa048107180 con 0x7fa048106610 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.567+0000 7fa046ffd640 1 --2- 172.21.15.43:0/3076773053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa048106a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa038009820 tx=0x7fa03802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572c9a0685b73965 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa045ffb640 1 -- 172.21.15.43:0/3076773053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa038037070 con 0x7fa048106610 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa045ffb640 1 -- 172.21.15.43:0/3076773053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fa038032890 con 0x7fa048106610 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa045ffb640 1 -- 172.21.15.43:0/3076773053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa038030550 con 0x7fa048106610 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa04d869640 1 -- 172.21.15.43:0/3076773053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 msgr2=0x7fa048106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa04d869640 1 --2- 172.21.15.43:0/3076773053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa048106a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa038009820 tx=0x7fa03802a270 comp rx=0 tx=0).stop 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa04d869640 1 -- 172.21.15.43:0/3076773053 shutdown_connections 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa04d869640 1 --2- 172.21.15.43:0/3076773053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa048106a10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa04d869640 1 -- 172.21.15.43:0/3076773053 >> 172.21.15.43:0/3076773053 conn(0x7fa048101f00 msgr2=0x7fa048104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa04d869640 1 -- 172.21.15.43:0/3076773053 shutdown_connections 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.568+0000 7fa04d869640 1 -- 172.21.15.43:0/3076773053 wait complete. 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.569+0000 7fa04d869640 1 Processor -- start 2024-09-13T13:24:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.569+0000 7fa04d869640 1 -- start start 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.569+0000 7fa04d869640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa04819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.569+0000 7fa04d869640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa04810aa90 con 0x7fa048106610 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.569+0000 7fa046ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa04819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.569+0000 7fa046ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa04819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43102/0 (socket says 172.21.15.43:43102) 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.569+0000 7fa046ffd640 1 -- 172.21.15.43:0/3744833072 learned_addr learned my addr 172.21.15.43:0/3744833072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.570+0000 7fa046ffd640 1 -- 172.21.15.43:0/3744833072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04819ab30 con 0x7fa048106610 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.570+0000 7fa046ffd640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa04819a5d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa038032860 tx=0x7fa038030a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.570+0000 7fa04c867640 1 -- 172.21.15.43:0/3744833072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa038037040 con 0x7fa048106610 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.570+0000 7fa04c867640 1 -- 172.21.15.43:0/3744833072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fa038040070 con 0x7fa048106610 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.570+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04819adc0 con 0x7fa048106610 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.570+0000 7fa04c867640 1 -- 172.21.15.43:0/3744833072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03803bd60 con 0x7fa048106610 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.570+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04819b780 con 0x7fa048106610 2024-09-13T13:24:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.571+0000 7fa04c867640 1 -- 172.21.15.43:0/3744833072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 51778+0+0 (secure 0 0 0) 0x7fa03803b430 con 0x7fa048106610 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.571+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04810ac10 con 0x7fa048106610 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.571+0000 7fa04c867640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa03003f700 0x7fa030041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.572+0000 7fa04c867640 1 -- 172.21.15.43:0/3744833072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa038074280 con 0x7fa048106610 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.572+0000 7fa0467fc640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa03003f700 0x7fa030041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.572+0000 7fa0467fc640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa03003f700 0x7fa030041bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0098c0 tx=0x7fa03c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.577+0000 7fa04c867640 1 -- 172.21.15.43:0/3744833072 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fa038030dc0 con 0x7fa048106610 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.859+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa048106a10 con 0x7fa048106610 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.861+0000 7fa04c867640 1 -- 172.21.15.43:0/3744833072 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fa038033cf0 con 0x7fa048106610 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-13T13:24:18.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.863+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa03003f700 msgr2=0x7fa030041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.863+0000 7fa04d869640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa03003f700 0x7fa030041bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0098c0 tx=0x7fa03c006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 msgr2=0x7fa04819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa04819a5d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa038032860 tx=0x7fa038030a80 comp rx=0 tx=0).stop 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 shutdown_connections 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa03003f700 0x7fa030041bc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 --2- 172.21.15.43:0/3744833072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa048106610 0x7fa04819a5d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 >> 172.21.15.43:0/3744833072 conn(0x7fa048101f00 msgr2=0x7fa048102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 shutdown_connections 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-13T13:24:17.864+0000 7fa04d869640 1 -- 172.21.15.43:0/3744833072 wait complete. 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-13T13:24:18.398 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.399 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-09-13T13:24:18.400 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.400 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-13T13:24:18.400 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:18.400 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-09-13T13:24:18.432 INFO:tasks.cephadm:Fetching config... 2024-09-13T13:24:18.432 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:24:18.432 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-13T13:24:18.449 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-13T13:24:18.449 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:24:18.449 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-13T13:24:18.505 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-13T13:24:18.505 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:24:18.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-13T13:24:18.571 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-13T13:24:18.572 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:24:18.572 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-13T13:24:18.629 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-13T13:24:18.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjLkD3SbEjMgMtLUMLCxIB8Rn9GPtg9iA4GqWze4u2DQlo5xzPcR+l/xxvqff5eDNGIESRenYicsQlcSd/zP91blQWVBszIe2+PmuWVaGJsGjEOqqqiAVcpORpSA9PueZQ/YtAskbF5yTwRQoRLGxosgjzg2PpsyIQbOVpJN6ss7grn7Gfdyh2jVlcs2csSgjhEEa5yk9Nca1Njr/qKx7u+1RCA4tN0lJ4TiyAVIl3XnTDcxISxAKQQqMx6M7Pue84H7HqEkTURCstdpwPjWJBmWFS9US0vR9r5MCUIy+5ebwX/I8uXk4jQV50JJwMj1VtQFUMNi/uQX8YIKVDnbdd+GtXOzZSxRpbDaMLK+12VtMKPk8McBW5+ZOVlYLR/CX/laTPsfRdVUgnYvcAJaNzxKQh/0QvxgiRTAJ0pgSfa2vsi93OkrA4JvlPLwFqiv3xq4FHAQ886w5BPMx99gcym2QLB7C/s5nWmzhJXi84noSbSGxuy3ACctXrBt+LvT0= ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T13:24:18.704 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjLkD3SbEjMgMtLUMLCxIB8Rn9GPtg9iA4GqWze4u2DQlo5xzPcR+l/xxvqff5eDNGIESRenYicsQlcSd/zP91blQWVBszIe2+PmuWVaGJsGjEOqqqiAVcpORpSA9PueZQ/YtAskbF5yTwRQoRLGxosgjzg2PpsyIQbOVpJN6ss7grn7Gfdyh2jVlcs2csSgjhEEa5yk9Nca1Njr/qKx7u+1RCA4tN0lJ4TiyAVIl3XnTDcxISxAKQQqMx6M7Pue84H7HqEkTURCstdpwPjWJBmWFS9US0vR9r5MCUIy+5ebwX/I8uXk4jQV50JJwMj1VtQFUMNi/uQX8YIKVDnbdd+GtXOzZSxRpbDaMLK+12VtMKPk8McBW5+ZOVlYLR/CX/laTPsfRdVUgnYvcAJaNzxKQh/0QvxgiRTAJ0pgSfa2vsi93OkrA4JvlPLwFqiv3xq4FHAQ886w5BPMx99gcym2QLB7C/s5nWmzhJXi84noSbSGxuy3ACctXrBt+LvT0= ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:18.716 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjLkD3SbEjMgMtLUMLCxIB8Rn9GPtg9iA4GqWze4u2DQlo5xzPcR+l/xxvqff5eDNGIESRenYicsQlcSd/zP91blQWVBszIe2+PmuWVaGJsGjEOqqqiAVcpORpSA9PueZQ/YtAskbF5yTwRQoRLGxosgjzg2PpsyIQbOVpJN6ss7grn7Gfdyh2jVlcs2csSgjhEEa5yk9Nca1Njr/qKx7u+1RCA4tN0lJ4TiyAVIl3XnTDcxISxAKQQqMx6M7Pue84H7HqEkTURCstdpwPjWJBmWFS9US0vR9r5MCUIy+5ebwX/I8uXk4jQV50JJwMj1VtQFUMNi/uQX8YIKVDnbdd+GtXOzZSxRpbDaMLK+12VtMKPk8McBW5+ZOVlYLR/CX/laTPsfRdVUgnYvcAJaNzxKQh/0QvxgiRTAJ0pgSfa2vsi93OkrA4JvlPLwFqiv3xq4FHAQ886w5BPMx99gcym2QLB7C/s5nWmzhJXi84noSbSGxuy3ACctXrBt+LvT0= ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T13:24:18.772 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjLkD3SbEjMgMtLUMLCxIB8Rn9GPtg9iA4GqWze4u2DQlo5xzPcR+l/xxvqff5eDNGIESRenYicsQlcSd/zP91blQWVBszIe2+PmuWVaGJsGjEOqqqiAVcpORpSA9PueZQ/YtAskbF5yTwRQoRLGxosgjzg2PpsyIQbOVpJN6ss7grn7Gfdyh2jVlcs2csSgjhEEa5yk9Nca1Njr/qKx7u+1RCA4tN0lJ4TiyAVIl3XnTDcxISxAKQQqMx6M7Pue84H7HqEkTURCstdpwPjWJBmWFS9US0vR9r5MCUIy+5ebwX/I8uXk4jQV50JJwMj1VtQFUMNi/uQX8YIKVDnbdd+GtXOzZSxRpbDaMLK+12VtMKPk8McBW5+ZOVlYLR/CX/laTPsfRdVUgnYvcAJaNzxKQh/0QvxgiRTAJ0pgSfa2vsi93OkrA4JvlPLwFqiv3xq4FHAQ886w5BPMx99gcym2QLB7C/s5nWmzhJXi84noSbSGxuy3ACctXrBt+LvT0= ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:18.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-13T13:24:18.982 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3744833072' entity='client.admin' 2024-09-13T13:24:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:24:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.514+0000 7fea79fde640 1 -- 172.21.15.43:0/4038389294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 msgr2=0x7fea74102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.514+0000 7fea79fde640 1 --2- 172.21.15.43:0/4038389294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea74102820 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea64009880 tx=0x7fea6402a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.514+0000 7fea79fde640 1 -- 172.21.15.43:0/4038389294 shutdown_connections 2024-09-13T13:24:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.514+0000 7fea79fde640 1 --2- 172.21.15.43:0/4038389294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea74102820 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.514+0000 7fea79fde640 1 -- 172.21.15.43:0/4038389294 >> 172.21.15.43:0/4038389294 conn(0x7fea740fdcd0 msgr2=0x7fea74100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea79fde640 1 -- 172.21.15.43:0/4038389294 shutdown_connections 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea79fde640 1 -- 172.21.15.43:0/4038389294 wait complete. 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea79fde640 1 Processor -- start 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea79fde640 1 -- start start 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea79fde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea741993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea79fde640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea741078f0 con 0x7fea74102420 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea741993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea741993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43122/0 (socket says 172.21.15.43:43122) 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.515+0000 7fea737fe640 1 -- 172.21.15.43:0/3477681140 learned_addr learned my addr 172.21.15.43:0/3477681140 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.516+0000 7fea737fe640 1 -- 172.21.15.43:0/3477681140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea74199920 con 0x7fea74102420 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.516+0000 7fea737fe640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea741993c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea64004dc0 tx=0x7fea64005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.516+0000 7fea717fa640 1 -- 172.21.15.43:0/3477681140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea6403f070 con 0x7fea74102420 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.516+0000 7fea717fa640 1 -- 172.21.15.43:0/3477681140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fea64003a40 con 0x7fea74102420 2024-09-13T13:24:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.516+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea74199bb0 con 0x7fea74102420 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.516+0000 7fea717fa640 1 -- 172.21.15.43:0/3477681140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea64035030 con 0x7fea74102420 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.516+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7419a570 con 0x7fea74102420 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.517+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea74108070 con 0x7fea74102420 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.518+0000 7fea717fa640 1 -- 172.21.15.43:0/3477681140 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 51778+0+0 (secure 0 0 0) 0x7fea64042070 con 0x7fea74102420 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.518+0000 7fea717fa640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fea6003f6a0 0x7fea60041b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.518+0000 7fea717fa640 1 -- 172.21.15.43:0/3477681140 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fea64074dd0 con 0x7fea74102420 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.518+0000 7fea6bfff640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fea6003f6a0 0x7fea60041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.518+0000 7fea6bfff640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fea6003f6a0 0x7fea60041b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098c0 tx=0x7fea5c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.523+0000 7fea717fa640 1 -- 172.21.15.43:0/3477681140 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fea6402f300 con 0x7fea74102420 2024-09-13T13:24:19.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.710+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fea7419ab40 con 0x7fea74102420 2024-09-13T13:24:19.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.713+0000 7fea717fa640 1 -- 172.21.15.43:0/3477681140 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fea64076030 con 0x7fea74102420 2024-09-13T13:24:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fea6003f6a0 msgr2=0x7fea60041b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fea6003f6a0 0x7fea60041b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098c0 tx=0x7fea5c006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 msgr2=0x7fea741993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea741993c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea64004dc0 tx=0x7fea64005090 comp rx=0 tx=0).stop 2024-09-13T13:24:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 shutdown_connections 2024-09-13T13:24:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fea6003f6a0 0x7fea60041b60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 --2- 172.21.15.43:0/3477681140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102420 0x7fea741993c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 >> 172.21.15.43:0/3477681140 conn(0x7fea740fdcd0 msgr2=0x7fea74100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 shutdown_connections 2024-09-13T13:24:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:19.716+0000 7fea79fde640 1 -- 172.21.15.43:0/3477681140 wait complete. 2024-09-13T13:24:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3477681140' entity='client.admin' 2024-09-13T13:24:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: mgrmap e14: y(active, since 12s) 2024-09-13T13:24:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"} : dispatch 2024-09-13T13:24:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:21.158 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-13T13:24:21.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-13T13:24:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.408+0000 7f84295f5640 1 -- 172.21.15.43:0/1065177982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 msgr2=0x7f8424075560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.408+0000 7f84295f5640 1 --2- 172.21.15.43:0/1065177982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424075560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f841c009880 tx=0x7f841c02a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.409+0000 7f84295f5640 1 -- 172.21.15.43:0/1065177982 shutdown_connections 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.409+0000 7f84295f5640 1 --2- 172.21.15.43:0/1065177982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424075560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.409+0000 7f84295f5640 1 -- 172.21.15.43:0/1065177982 >> 172.21.15.43:0/1065177982 conn(0x7f84240fdcb0 msgr2=0x7f8424100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.409+0000 7f84295f5640 1 -- 172.21.15.43:0/1065177982 shutdown_connections 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.409+0000 7f84295f5640 1 -- 172.21.15.43:0/1065177982 wait complete. 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.409+0000 7f84295f5640 1 Processor -- start 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.409+0000 7f84295f5640 1 -- start start 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f84295f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f84295f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8424109ac0 con 0x7f8424076f00 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f8422ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f8422ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43138/0 (socket says 172.21.15.43:43138) 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f8422ffd640 1 -- 172.21.15.43:0/2928454712 learned_addr learned my addr 172.21.15.43:0/2928454712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f8422ffd640 1 -- 172.21.15.43:0/2928454712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84241998d0 con 0x7f8424076f00 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f8422ffd640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424199370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f841c004dc0 tx=0x7f841c005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f841bfff640 1 -- 172.21.15.43:0/2928454712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c03f070 con 0x7f8424076f00 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f841bfff640 1 -- 172.21.15.43:0/2928454712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f841c003a40 con 0x7f8424076f00 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8424199b60 con 0x7f8424076f00 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f841bfff640 1 -- 172.21.15.43:0/2928454712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c035030 con 0x7f8424076f00 2024-09-13T13:24:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.410+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842419a520 con 0x7f8424076f00 2024-09-13T13:24:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.411+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842410a240 con 0x7f8424076f00 2024-09-13T13:24:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.411+0000 7f841bfff640 1 -- 172.21.15.43:0/2928454712 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7f841c042070 con 0x7f8424076f00 2024-09-13T13:24:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.412+0000 7f841bfff640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f841003f7a0 0x7f8410041c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.412+0000 7f841bfff640 1 -- 172.21.15.43:0/2928454712 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f841c074ca0 con 0x7f8424076f00 2024-09-13T13:24:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.412+0000 7f84227fc640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f841003f7a0 0x7f8410041c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.412+0000 7f84227fc640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f841003f7a0 0x7f8410041c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f840c0098c0 tx=0x7f840c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.416+0000 7f841bfff640 1 -- 172.21.15.43:0/2928454712 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f841c02f300 con 0x7f8424076f00 2024-09-13T13:24:22.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.606+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f842419b260 con 0x7f841003f7a0 2024-09-13T13:24:22.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.609+0000 7f841bfff640 1 -- 172.21.15.43:0/2928454712 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f842419b260 con 0x7f841003f7a0 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f841003f7a0 msgr2=0x7f8410041c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f841003f7a0 0x7f8410041c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f840c0098c0 tx=0x7f840c006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 msgr2=0x7f8424199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424199370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f841c004dc0 tx=0x7f841c005090 comp rx=0 tx=0).stop 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 shutdown_connections 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f841003f7a0 0x7f8410041c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 --2- 172.21.15.43:0/2928454712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8424076f00 0x7f8424199370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.611+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 >> 172.21.15.43:0/2928454712 conn(0x7f84240fdcb0 msgr2=0x7f84240fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.612+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 shutdown_connections 2024-09-13T13:24:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:22.612+0000 7f84295f5640 1 -- 172.21.15.43:0/2928454712 wait complete. 2024-09-13T13:24:23.304 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-09-13T13:24:23.305 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:24:23.305 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-09-13T13:24:23.325 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:24:23.325 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:24:23.383 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-09-13T13:24:23.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch host add smithi189 2024-09-13T13:24:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:24:23.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:23.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.092+0000 7f7e793fa640 1 -- 172.21.15.43:0/2981187026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 msgr2=0x7f7e74103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.092+0000 7f7e793fa640 1 --2- 172.21.15.43:0/2981187026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74103a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009880 tx=0x7f7e6402a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.092+0000 7f7e793fa640 1 -- 172.21.15.43:0/2981187026 shutdown_connections 2024-09-13T13:24:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.092+0000 7f7e793fa640 1 --2- 172.21.15.43:0/2981187026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74103a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.092+0000 7f7e793fa640 1 -- 172.21.15.43:0/2981187026 >> 172.21.15.43:0/2981187026 conn(0x7f7e740fd0c0 msgr2=0x7f7e740ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.092+0000 7f7e793fa640 1 -- 172.21.15.43:0/2981187026 shutdown_connections 2024-09-13T13:24:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.092+0000 7f7e793fa640 1 -- 172.21.15.43:0/2981187026 wait complete. 2024-09-13T13:24:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.093+0000 7f7e793fa640 1 Processor -- start 2024-09-13T13:24:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.093+0000 7f7e793fa640 1 -- start start 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e793fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e793fa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7410bef0 con 0x7f7e74103680 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74101330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43158/0 (socket says 172.21.15.43:43158) 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e72ffd640 1 -- 172.21.15.43:0/1703324452 learned_addr learned my addr 172.21.15.43:0/1703324452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e72ffd640 1 -- 172.21.15.43:0/1703324452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e741030d0 con 0x7f7e74103680 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e72ffd640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74101330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e64005ab0 tx=0x7f7e64003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e5bfff640 1 -- 172.21.15.43:0/1703324452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e6403f070 con 0x7f7e74103680 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e5bfff640 1 -- 172.21.15.43:0/1703324452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7e6403bbf0 con 0x7f7e74103680 2024-09-13T13:24:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e74101870 con 0x7f7e74103680 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e5bfff640 1 -- 172.21.15.43:0/1703324452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64035030 con 0x7f7e74103680 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.094+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e74101df0 con 0x7f7e74103680 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.095+0000 7f7e5bfff640 1 -- 172.21.15.43:0/1703324452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7f7e64042070 con 0x7f7e74103680 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.095+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7406b530 con 0x7f7e74103680 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.096+0000 7f7e5bfff640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7e5c03f7a0 0x7f7e5c041c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.096+0000 7f7e5bfff640 1 -- 172.21.15.43:0/1703324452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e64074a90 con 0x7f7e74103680 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.096+0000 7f7e727fc640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7e5c03f7a0 0x7f7e5c041c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.096+0000 7f7e727fc640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7e5c03f7a0 0x7f7e5c041c60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098c0 tx=0x7f7e68006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.100+0000 7f7e5bfff640 1 -- 172.21.15.43:0/1703324452 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f7e64030e50 con 0x7f7e74103680 2024-09-13T13:24:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:24.290+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7f7e74103ae0 con 0x7f7e5c03f7a0 2024-09-13T13:24:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:24 smithi043 ceph-mon[30106]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:24:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:24 smithi043 ceph-mon[30106]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:24:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:25 smithi043 ceph-mon[30106]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:26 smithi043 ceph-mon[30106]: Deploying cephadm binary to smithi189 2024-09-13T13:24:28.021 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-09-13T13:24:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.019+0000 7f7e5bfff640 1 -- 172.21.15.43:0/1703324452 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7e74103ae0 con 0x7f7e5c03f7a0 2024-09-13T13:24:28.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7e5c03f7a0 msgr2=0x7f7e5c041c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7e5c03f7a0 0x7f7e5c041c60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098c0 tx=0x7f7e68006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 msgr2=0x7f7e74101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74101330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e64005ab0 tx=0x7f7e64003d80 comp rx=0 tx=0).stop 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 shutdown_connections 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7e5c03f7a0 0x7f7e5c041c60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 --2- 172.21.15.43:0/1703324452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103680 0x7f7e74101330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.022+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 >> 172.21.15.43:0/1703324452 conn(0x7f7e740fd0c0 msgr2=0x7f7e74106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.023+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 shutdown_connections 2024-09-13T13:24:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:28.023+0000 7f7e793fa640 1 -- 172.21.15.43:0/1703324452 wait complete. 2024-09-13T13:24:28.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T13:24:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:24:28.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:28 smithi043 ceph-mon[30106]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:28.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:28.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:28.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:28.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:29.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 -- 172.21.15.43:0/890941306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 msgr2=0x7fa060103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:29.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 --2- 172.21.15.43:0/890941306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060103a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa050009880 tx=0x7fa05002a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 -- 172.21.15.43:0/890941306 shutdown_connections 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 --2- 172.21.15.43:0/890941306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060103a90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 -- 172.21.15.43:0/890941306 >> 172.21.15.43:0/890941306 conn(0x7fa0600fd0f0 msgr2=0x7fa0600ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 -- 172.21.15.43:0/890941306 shutdown_connections 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 -- 172.21.15.43:0/890941306 wait complete. 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.305+0000 7fa064b2e640 1 Processor -- start 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa064b2e640 1 -- start start 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa064b2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa064b2e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010bf20 con 0x7fa0601036b0 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa05e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa05e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57704/0 (socket says 172.21.15.43:57704) 2024-09-13T13:24:29.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa05e59c640 1 -- 172.21.15.43:0/2240602313 learned_addr learned my addr 172.21.15.43:0/2240602313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa05e59c640 1 -- 172.21.15.43:0/2240602313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa060103100 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa05e59c640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060101360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa050005ab0 tx=0x7fa050003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa0577fe640 1 -- 172.21.15.43:0/2240602313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa05003f070 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa0577fe640 1 -- 172.21.15.43:0/2240602313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fa05003bbf0 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0601018a0 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa0577fe640 1 -- 172.21.15.43:0/2240602313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050035030 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.306+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa060101e20 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.307+0000 7fa0577fe640 1 -- 172.21.15.43:0/2240602313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7fa050042070 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.307+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06006b530 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.308+0000 7fa0577fe640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa04c03f7a0 0x7fa04c041c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.308+0000 7fa0577fe640 1 -- 172.21.15.43:0/2240602313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa050074af0 con 0x7fa0601036b0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.308+0000 7fa057fff640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa04c03f7a0 0x7fa04c041c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:29.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.308+0000 7fa057fff640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa04c03f7a0 0x7fa04c041c60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098c0 tx=0x7fa048006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:29.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.312+0000 7fa0577fe640 1 -- 172.21.15.43:0/2240602313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fa050031400 con 0x7fa0601036b0 2024-09-13T13:24:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.508+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa060103b10 con 0x7fa04c03f7a0 2024-09-13T13:24:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.509+0000 7fa0577fe640 1 -- 172.21.15.43:0/2240602313 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fa060103b10 con 0x7fa04c03f7a0 2024-09-13T13:24:29.510 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:24:29.510 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-09-13T13:24:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa04c03f7a0 msgr2=0x7fa04c041c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa04c03f7a0 0x7fa04c041c60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098c0 tx=0x7fa048006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 msgr2=0x7fa060101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060101360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa050005ab0 tx=0x7fa050003d80 comp rx=0 tx=0).stop 2024-09-13T13:24:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 shutdown_connections 2024-09-13T13:24:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa04c03f7a0 0x7fa04c041c60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 --2- 172.21.15.43:0/2240602313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0601036b0 0x7fa060101360 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 >> 172.21.15.43:0/2240602313 conn(0x7fa0600fd0f0 msgr2=0x7fa060106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 shutdown_connections 2024-09-13T13:24:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:29.512+0000 7fa064b2e640 1 -- 172.21.15.43:0/2240602313 wait complete. 2024-09-13T13:24:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:29 smithi043 ceph-mon[30106]: Added host smithi189 2024-09-13T13:24:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:30.146 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-13T13:24:30.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-13T13:24:30.398 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:24:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:30 smithi043 ceph-mon[30106]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:30.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:30 smithi043 ceph-mon[30106]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:24:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.812+0000 7ff284bb2640 1 -- 172.21.15.43:0/2934659416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 msgr2=0x7ff280101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.812+0000 7ff284bb2640 1 --2- 172.21.15.43:0/2934659416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff280101a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff270009880 tx=0x7ff27002a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.812+0000 7ff284bb2640 1 -- 172.21.15.43:0/2934659416 shutdown_connections 2024-09-13T13:24:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.812+0000 7ff284bb2640 1 --2- 172.21.15.43:0/2934659416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff280101a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.812+0000 7ff284bb2640 1 -- 172.21.15.43:0/2934659416 >> 172.21.15.43:0/2934659416 conn(0x7ff2800fd0f0 msgr2=0x7ff2800ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.812+0000 7ff284bb2640 1 -- 172.21.15.43:0/2934659416 shutdown_connections 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.812+0000 7ff284bb2640 1 -- 172.21.15.43:0/2934659416 wait complete. 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff284bb2640 1 Processor -- start 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff284bb2640 1 -- start start 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff284bb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff2801960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff284bb2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff28010adc0 con 0x7ff2801016a0 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff27e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff2801960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff27e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff2801960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57722/0 (socket says 172.21.15.43:57722) 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff27e59c640 1 -- 172.21.15.43:0/2775734466 learned_addr learned my addr 172.21.15.43:0/2775734466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff27e59c640 1 -- 172.21.15.43:0/2775734466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28006bb20 con 0x7ff2801016a0 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.813+0000 7ff27e59c640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff2801960c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff270005d90 tx=0x7ff270003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.814+0000 7ff2777fe640 1 -- 172.21.15.43:0/2775734466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27003f070 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.814+0000 7ff2777fe640 1 -- 172.21.15.43:0/2775734466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ff27003bbf0 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.814+0000 7ff2777fe640 1 -- 172.21.15.43:0/2775734466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff270035030 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.814+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff280196600 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.814+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff280196be0 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.815+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff280102950 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.815+0000 7ff2777fe640 1 -- 172.21.15.43:0/2775734466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7ff270042070 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.815+0000 7ff2777fe640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff26c03f7a0 0x7ff26c041c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.815+0000 7ff2777fe640 1 -- 172.21.15.43:0/2775734466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff270074bd0 con 0x7ff2801016a0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.815+0000 7ff277fff640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff26c03f7a0 0x7ff26c041c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.815+0000 7ff277fff640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff26c03f7a0 0x7ff26c041c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098c0 tx=0x7ff268006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:30.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:30.820+0000 7ff2777fe640 1 -- 172.21.15.43:0/2775734466 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff27004a260 con 0x7ff2801016a0 2024-09-13T13:24:31.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.009+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff280101b00 con 0x7ff2801016a0 2024-09-13T13:24:31.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.617+0000 7ff2777fe640 1 -- 172.21.15.43:0/2775734466 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff27003a3c0 con 0x7ff2801016a0 2024-09-13T13:24:31.618 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff26c03f7a0 msgr2=0x7ff26c041c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff26c03f7a0 0x7ff26c041c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098c0 tx=0x7ff268006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 msgr2=0x7ff2801960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff2801960c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff270005d90 tx=0x7ff270003d60 comp rx=0 tx=0).stop 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 shutdown_connections 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff26c03f7a0 0x7ff26c041c60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 --2- 172.21.15.43:0/2775734466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2801016a0 0x7ff2801960c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 >> 172.21.15.43:0/2775734466 conn(0x7ff2800fd0f0 msgr2=0x7ff280104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 shutdown_connections 2024-09-13T13:24:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:31.620+0000 7ff284bb2640 1 -- 172.21.15.43:0/2775734466 wait complete. 2024-09-13T13:24:31.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2775734466' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-13T13:24:32.261 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-09-13T13:24:32.261 INFO:tasks.cephadm:Adding mon.c on smithi043 2024-09-13T13:24:32.261 INFO:tasks.cephadm:Adding mon.b on smithi189 2024-09-13T13:24:32.261 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi189:172.21.15.189=b' 2024-09-13T13:24:32.506 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2775734466' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:32.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"} : dispatch 2024-09-13T13:24:32.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:32.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:32.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.956+0000 7eff90608640 1 -- 172.21.15.189:0/4059531753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 msgr2=0x7eff88102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:32.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.956+0000 7eff90608640 1 --2- 172.21.15.189:0/4059531753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff88102cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff84009880 tx=0x7eff8402a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:32.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.956+0000 7eff90608640 1 -- 172.21.15.189:0/4059531753 shutdown_connections 2024-09-13T13:24:32.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.956+0000 7eff90608640 1 --2- 172.21.15.189:0/4059531753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff88102cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:32.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.956+0000 7eff90608640 1 -- 172.21.15.189:0/4059531753 >> 172.21.15.189:0/4059531753 conn(0x7eff88078270 msgr2=0x7eff88078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:32.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.956+0000 7eff90608640 1 -- 172.21.15.189:0/4059531753 shutdown_connections 2024-09-13T13:24:32.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.956+0000 7eff90608640 1 -- 172.21.15.189:0/4059531753 wait complete. 2024-09-13T13:24:32.959 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.957+0000 7eff90608640 1 Processor -- start 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.957+0000 7eff90608640 1 -- start start 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.957+0000 7eff90608640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff880754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.957+0000 7eff90608640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff8810bea0 con 0x7eff88102910 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff8e3a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff880754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff8e3a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff880754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.189:43948/0 (socket says 172.21.15.189:43948) 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff8e3a4640 1 -- 172.21.15.189:0/256128611 learned_addr learned my addr 172.21.15.189:0/256128611 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff8e3a4640 1 -- 172.21.15.189:0/256128611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff88077240 con 0x7eff88102910 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff8e3a4640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff880754a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff84005d90 tx=0x7eff84003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff7f7fe640 1 -- 172.21.15.189:0/256128611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff8403f070 con 0x7eff88102910 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff7f7fe640 1 -- 172.21.15.189:0/256128611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7eff8403bbf0 con 0x7eff88102910 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff880759e0 con 0x7eff88102910 2024-09-13T13:24:32.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff7f7fe640 1 -- 172.21.15.189:0/256128611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff84035030 con 0x7eff88102910 2024-09-13T13:24:32.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.958+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff88075ee0 con 0x7eff88102910 2024-09-13T13:24:32.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.959+0000 7eff7f7fe640 1 -- 172.21.15.189:0/256128611 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7eff84042070 con 0x7eff88102910 2024-09-13T13:24:32.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.959+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff88103a10 con 0x7eff88102910 2024-09-13T13:24:32.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.959+0000 7eff7f7fe640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7eff7403f7a0 0x7eff74041c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:32.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.959+0000 7eff7f7fe640 1 -- 172.21.15.189:0/256128611 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7eff84074aa0 con 0x7eff88102910 2024-09-13T13:24:32.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.960+0000 7eff8dba3640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7eff7403f7a0 0x7eff74041c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:32.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.960+0000 7eff8dba3640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7eff7403f7a0 0x7eff74041c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff780098c0 tx=0x7eff78006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:32.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:32.965+0000 7eff7f7fe640 1 -- 172.21.15.189:0/256128611 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7eff8403bda0 con 0x7eff88102910 2024-09-13T13:24:33.154 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.153+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi189:172.21.15.189=b", "target": ["mon-mgr", ""]}) -- 0x7eff880767a0 con 0x7eff7403f7a0 2024-09-13T13:24:33.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.159+0000 7eff7f7fe640 1 -- 172.21.15.189:0/256128611 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7eff880767a0 con 0x7eff7403f7a0 2024-09-13T13:24:33.160 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7eff7403f7a0 msgr2=0x7eff74041c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7eff7403f7a0 0x7eff74041c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff780098c0 tx=0x7eff78006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 msgr2=0x7eff880754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff880754a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff84005d90 tx=0x7eff84003d60 comp rx=0 tx=0).stop 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 shutdown_connections 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7eff7403f7a0 0x7eff74041c60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 --2- 172.21.15.189:0/256128611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff88102910 0x7eff880754a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:33.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 >> 172.21.15.189:0/256128611 conn(0x7eff88078270 msgr2=0x7eff88104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:33.164 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 shutdown_connections 2024-09-13T13:24:33.164 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:33.161+0000 7eff90608640 1 -- 172.21.15.189:0/256128611 wait complete. 2024-09-13T13:24:33.744 DEBUG:teuthology.orchestra.run.smithi043:mon.c> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.c.service 2024-09-13T13:24:33.747 DEBUG:teuthology.orchestra.run.smithi189:mon.b> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.b.service 2024-09-13T13:24:33.750 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T13:24:33.751 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:33.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:33.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:33.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:33.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:34.392 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:34.852 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.831+0000 7fab88b7f640 1 -- 172.21.15.189:0/3210813841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 msgr2=0x7fab84101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:34.852 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.831+0000 7fab88b7f640 1 --2- 172.21.15.189:0/3210813841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab84101a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab74009880 tx=0x7fab7402a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:34.852 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.831+0000 7fab88b7f640 1 -- 172.21.15.189:0/3210813841 shutdown_connections 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.831+0000 7fab88b7f640 1 --2- 172.21.15.189:0/3210813841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab84101a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.831+0000 7fab88b7f640 1 -- 172.21.15.189:0/3210813841 >> 172.21.15.189:0/3210813841 conn(0x7fab840fd0f0 msgr2=0x7fab840ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.831+0000 7fab88b7f640 1 -- 172.21.15.189:0/3210813841 shutdown_connections 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.831+0000 7fab88b7f640 1 -- 172.21.15.189:0/3210813841 wait complete. 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.832+0000 7fab88b7f640 1 Processor -- start 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.832+0000 7fab88b7f640 1 -- start start 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.832+0000 7fab88b7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab841963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.832+0000 7fab88b7f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab8410adc0 con 0x7fab841016a0 2024-09-13T13:24:34.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab8259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab841963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab8259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab841963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.189:43974/0 (socket says 172.21.15.189:43974) 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab8259c640 1 -- 172.21.15.189:0/1119094768 learned_addr learned my addr 172.21.15.189:0/1119094768 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab8259c640 1 -- 172.21.15.189:0/1119094768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab841968f0 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab8259c640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab841963b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab74005d90 tx=0x7fab74003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab7b7fe640 1 -- 172.21.15.189:0/1119094768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab7403f070 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab7b7fe640 1 -- 172.21.15.189:0/1119094768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fab7403bbf0 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab84196be0 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab7b7fe640 1 -- 172.21.15.189:0/1119094768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab74035030 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.833+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8419bc90 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.834+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab84102950 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.834+0000 7fab7b7fe640 1 -- 172.21.15.189:0/1119094768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7fab74042070 con 0x7fab841016a0 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.835+0000 7fab7b7fe640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fab7003f740 0x7fab70041c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:34.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.835+0000 7fab7b7fe640 1 -- 172.21.15.189:0/1119094768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fab74074ae0 con 0x7fab841016a0 2024-09-13T13:24:34.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.835+0000 7fab7bfff640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fab7003f740 0x7fab70041c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:34.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.835+0000 7fab7bfff640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fab7003f740 0x7fab70041c00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0098c0 tx=0x7fab6c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:34.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:34.840+0000 7fab7b7fe640 1 -- 172.21.15.189:0/1119094768 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fab7403a3c0 con 0x7fab841016a0 2024-09-13T13:24:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:34 smithi043 ceph-mon[30106]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi189:172.21.15.189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:34 smithi043 ceph-mon[30106]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi189:172.21.15.189=b;count:3 2024-09-13T13:24:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:34 smithi043 ceph-mon[30106]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:34 smithi043 ceph-mon[30106]: Deploying daemon mon.b on smithi189 2024-09-13T13:24:35.087 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.085+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fab841974e0 con 0x7fab841016a0 2024-09-13T13:24:35.087 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.086+0000 7fab7b7fe640 1 -- 172.21.15.189:0/1119094768 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fab7403bda0 con 0x7fab841016a0 2024-09-13T13:24:35.088 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:24:35.088 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","modified":"2024-09-13T13:23:08.898338Z","created":"2024-09-13T13:23:08.898338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-13T13:24:35.088 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-09-13T13:24:35.089 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fab7003f740 msgr2=0x7fab70041c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:35.089 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fab7003f740 0x7fab70041c00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0098c0 tx=0x7fab6c006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 msgr2=0x7fab841963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab841963b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab74005d90 tx=0x7fab74003d60 comp rx=0 tx=0).stop 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 shutdown_connections 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fab7003f740 0x7fab70041c00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 --2- 172.21.15.189:0/1119094768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab841016a0 0x7fab841963b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 >> 172.21.15.189:0/1119094768 conn(0x7fab840fd0f0 msgr2=0x7fab84104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 shutdown_connections 2024-09-13T13:24:35.090 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:35.088+0000 7fab88b7f640 1 -- 172.21.15.189:0/1119094768 wait complete. 2024-09-13T13:24:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/1119094768' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:36.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T13:24:36.753 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T13:24:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:36 smithi043 ceph-mon[30106]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:37.123 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:24:37.917 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.909+0000 7fa4afed9640 1 -- 172.21.15.189:0/1095105688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 msgr2=0x7fa4a8101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.909+0000 7fa4afed9640 1 --2- 172.21.15.189:0/1095105688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8101b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009880 tx=0x7fa4a402a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.909+0000 7fa4afed9640 1 -- 172.21.15.189:0/1095105688 shutdown_connections 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.909+0000 7fa4afed9640 1 --2- 172.21.15.189:0/1095105688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8101b80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.909+0000 7fa4afed9640 1 -- 172.21.15.189:0/1095105688 >> 172.21.15.189:0/1095105688 conn(0x7fa4a80fd230 msgr2=0x7fa4a80ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.909+0000 7fa4afed9640 1 -- 172.21.15.189:0/1095105688 shutdown_connections 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.909+0000 7fa4afed9640 1 -- 172.21.15.189:0/1095105688 wait complete. 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.910+0000 7fa4afed9640 1 Processor -- start 2024-09-13T13:24:37.932 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.910+0000 7fa4afed9640 1 -- start start 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.910+0000 7fa4afed9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.910+0000 7fa4afed9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810afd0 con 0x7fa4a81017a0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa4adc75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa4adc75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.189:43992/0 (socket says 172.21.15.189:43992) 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa4adc75640 1 -- 172.21.15.189:0/4143819412 learned_addr learned my addr 172.21.15.189:0/4143819412 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa4adc75640 1 -- 172.21.15.189:0/4143819412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a8196bc0 con 0x7fa4a81017a0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa4adc75640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8196680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005d90 tx=0x7fa4a4003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa49effd640 1 -- 172.21.15.189:0/4143819412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4a403f070 con 0x7fa4a81017a0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a8196eb0 con 0x7fa4a81017a0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa49effd640 1 -- 172.21.15.189:0/4143819412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fa4a403bbf0 con 0x7fa4a81017a0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa49effd640 1 -- 172.21.15.189:0/4143819412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4a4035030 con 0x7fa4a81017a0 2024-09-13T13:24:37.933 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.911+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a819bf60 con 0x7fa4a81017a0 2024-09-13T13:24:37.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.912+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a8102a50 con 0x7fa4a81017a0 2024-09-13T13:24:37.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.912+0000 7fa49effd640 1 -- 172.21.15.189:0/4143819412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7fa4a4042070 con 0x7fa4a81017a0 2024-09-13T13:24:37.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.913+0000 7fa49effd640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa49403f740 0x7fa494041c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:37.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.913+0000 7fa49effd640 1 -- 172.21.15.189:0/4143819412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa4a4074b40 con 0x7fa4a81017a0 2024-09-13T13:24:37.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.913+0000 7fa4ad474640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa49403f740 0x7fa494041c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:37.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.913+0000 7fa4ad474640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa49403f740 0x7fa494041c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4980098c0 tx=0x7fa498006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:37.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:37.918+0000 7fa49effd640 1 -- 172.21.15.189:0/4143819412 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fa4a403a3c0 con 0x7fa4a81017a0 2024-09-13T13:24:38.202 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.200+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa4a8101c00 con 0x7fa4a81017a0 2024-09-13T13:24:38.202 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.201+0000 7fa49effd640 1 -- 172.21.15.189:0/4143819412 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fa4a404a260 con 0x7fa4a81017a0 2024-09-13T13:24:38.202 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:24:38.203 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","modified":"2024-09-13T13:23:08.898338Z","created":"2024-09-13T13:23:08.898338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-13T13:24:38.203 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.203+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa49403f740 msgr2=0x7fa494041c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.203+0000 7fa4afed9640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa49403f740 0x7fa494041c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4980098c0 tx=0x7fa498006db0 comp rx=0 tx=0).stop 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.203+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 msgr2=0x7fa4a8196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.203+0000 7fa4afed9640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8196680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005d90 tx=0x7fa4a4003d60 comp rx=0 tx=0).stop 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.204+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 shutdown_connections 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.204+0000 7fa4afed9640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fa49403f740 0x7fa494041c00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.204+0000 7fa4afed9640 1 --2- 172.21.15.189:0/4143819412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a81017a0 0x7fa4a8196680 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.204+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 >> 172.21.15.189:0/4143819412 conn(0x7fa4a80fd230 msgr2=0x7fa4a8104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.204+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 shutdown_connections 2024-09-13T13:24:38.205 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:38.204+0000 7fa4afed9640 1 -- 172.21.15.189:0/4143819412 wait complete. 2024-09-13T13:24:38.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:38 smithi043 ceph-mon[30106]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/4143819412' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:39 smithi189 systemd[1]: Started Ceph mon.b for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:24:40.000 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T13:24:40.001 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T13:24:40.266 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:24:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.295+0000 7fc0653f4640 1 -- 172.21.15.189:0/3893235755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 msgr2=0x7fc0480059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.295+0000 7fc0653f4640 1 --2- 172.21.15.189:0/3893235755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0480059c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc060102620 tx=0x7fc05002c3f0 comp rx=0 tx=0).stop 2024-09-13T13:24:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.295+0000 7fc0653f4640 1 -- 172.21.15.189:0/3893235755 shutdown_connections 2024-09-13T13:24:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.295+0000 7fc0653f4640 1 --2- 172.21.15.189:0/3893235755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0480059c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.295+0000 7fc0653f4640 1 -- 172.21.15.189:0/3893235755 >> 172.21.15.189:0/3893235755 conn(0x7fc0600fe340 msgr2=0x7fc060100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.295+0000 7fc0653f4640 1 -- 172.21.15.189:0/3893235755 shutdown_connections 2024-09-13T13:24:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.295+0000 7fc0653f4640 1 -- 172.21.15.189:0/3893235755 wait complete. 2024-09-13T13:24:44.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.296+0000 7fc0653f4640 1 Processor -- start 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc0653f4640 1 -- start start 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc0653f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0601998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc0653f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 0x7fc06019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc0653f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc060109bb0 con 0x7fc048003530 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc0653f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc060109a30 con 0x7fc060199e20 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05e7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 0x7fc06019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0601998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0601998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.189:46516/0 (socket says 172.21.15.189:46516) 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05effd640 1 -- 172.21.15.189:0/1559803067 learned_addr learned my addr 172.21.15.189:0/1559803067 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05e7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 0x7fc06019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:47346/0 (socket says 172.21.15.189:47346) 2024-09-13T13:24:44.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05e7fc640 1 -- 172.21.15.189:0/1559803067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 msgr2=0x7fc06019f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T13:24:44.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05e7fc640 1 -- 172.21.15.189:0/1559803067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 msgr2=0x7fc06019f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T13:24:44.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05e7fc640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 0x7fc06019f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T13:24:44.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05e7fc640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 0x7fc06019f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:24:44.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05effd640 1 -- 172.21.15.189:0/1559803067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 msgr2=0x7fc06019f1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:24:44.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05effd640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 0x7fc06019f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:44.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.297+0000 7fc05effd640 1 -- 172.21.15.189:0/1559803067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06019a4a0 con 0x7fc048003530 2024-09-13T13:24:44.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.298+0000 7fc05effd640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0601998e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc050017090 tx=0x7fc050012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.298+0000 7fc047fff640 1 -- 172.21.15.189:0/1559803067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc050012b40 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.298+0000 7fc047fff640 1 -- 172.21.15.189:0/1559803067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc0500054a0 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.298+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06019f760 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.298+0000 7fc047fff640 1 -- 172.21.15.189:0/1559803067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc050004620 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.298+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06019fd20 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.299+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06019ffd0 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.299+0000 7fc047fff640 1 -- 172.21.15.189:0/1559803067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7fc050044020 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.300+0000 7fc047fff640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc04c03f530 0x7fc04c0419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.300+0000 7fc047fff640 1 -- 172.21.15.189:0/1559803067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc050076710 con 0x7fc048003530 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.300+0000 7fc05e7fc640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc04c03f530 0x7fc04c0419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:44.303 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.300+0000 7fc05e7fc640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc04c03f530 0x7fc04c0419f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:44.306 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.304+0000 7fc047fff640 1 -- 172.21.15.189:0/1559803067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc05004b020 con 0x7fc048003530 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: mon.a calling monitor election 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: mon.b calling monitor election 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: monmap epoch 2 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: last_changed 2024-09-13T13:24:39.144489+0000 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: min_mon_release 19 (squid) 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: election_strategy: 1 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: fsmap 2024-09-13T13:24:44.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:44.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: mgrmap e14: y(active, since 37s) 2024-09-13T13:24:44.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: overall HEALTH_OK 2024-09-13T13:24:44.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:44.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:44.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:44.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:44.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:44 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:44.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.556+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc06019aa10 con 0x7fc048003530 2024-09-13T13:24:44.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.556+0000 7fc047fff640 1 -- 172.21.15.189:0/1559803067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc05003d440 con 0x7fc048003530 2024-09-13T13:24:44.559 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:24:44.559 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","modified":"2024-09-13T13:24:39.144489Z","created":"2024-09-13T13:23:08.898338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T13:24:44.559 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: mon.a calling monitor election 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: mon.b calling monitor election 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:44.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: monmap epoch 2 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: last_changed 2024-09-13T13:24:39.144489+0000 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: min_mon_release 19 (squid) 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: election_strategy: 1 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: fsmap 2024-09-13T13:24:44.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:44.562 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: mgrmap e14: y(active, since 37s) 2024-09-13T13:24:44.562 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: overall HEALTH_OK 2024-09-13T13:24:44.562 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:44.562 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:44.562 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:44.562 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:44.562 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:44 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc04c03f530 msgr2=0x7fc04c0419f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc04c03f530 0x7fc04c0419f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054008040 comp rx=0 tx=0).stop 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 msgr2=0x7fc0601998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0601998e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc050017090 tx=0x7fc050012680 comp rx=0 tx=0).stop 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 shutdown_connections 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc04c03f530 0x7fc04c0419f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc060199e20 0x7fc06019f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 --2- 172.21.15.189:0/1559803067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc048003530 0x7fc0601998e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 >> 172.21.15.189:0/1559803067 conn(0x7fc0600fe340 msgr2=0x7fc060104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 shutdown_connections 2024-09-13T13:24:44.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:44.559+0000 7fc0653f4640 1 -- 172.21.15.189:0/1559803067 wait complete. 2024-09-13T13:24:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:45 smithi189 ceph-mon[32912]: Deploying daemon mon.c on smithi043 2024-09-13T13:24:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/1559803067' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:45 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:45 smithi043 ceph-mon[30106]: Deploying daemon mon.c on smithi043 2024-09-13T13:24:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/1559803067' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:45 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:46.143 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T13:24:46.144 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T13:24:46.414 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:24:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:46 smithi043 ceph-mon[30106]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:46.535 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:46 smithi189 ceph-mon[32912]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:46.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- 172.21.15.189:0/2522279209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 msgr2=0x7f1564101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:46.833 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 --2- 172.21.15.189:0/2522279209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 0x7f1564101b80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1554009880 tx=0x7f155402a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:46.833 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- 172.21.15.189:0/2522279209 shutdown_connections 2024-09-13T13:24:46.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 --2- 172.21.15.189:0/2522279209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 0x7f1564101b80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:46.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- 172.21.15.189:0/2522279209 >> 172.21.15.189:0/2522279209 conn(0x7f15640fd230 msgr2=0x7f15640ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:46.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- 172.21.15.189:0/2522279209 shutdown_connections 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- 172.21.15.189:0/2522279209 wait complete. 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 Processor -- start 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- start start 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 0x7f1564198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 0x7f1564198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f156410afd0 con 0x7f15641017a0 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f156a7e1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f156410ae50 con 0x7f1564198b60 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f15637fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 0x7f1564198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f15637fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 0x7f1564198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:47362/0 (socket says 172.21.15.189:47362) 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f15637fe640 1 -- 172.21.15.189:0/2920271246 learned_addr learned my addr 172.21.15.189:0/2920271246 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.829+0000 7f1563fff640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 0x7f1564198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:46.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f15637fe640 1 -- 172.21.15.189:0/2920271246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 msgr2=0x7f1564198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f15637fe640 1 -- 172.21.15.189:0/2920271246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 msgr2=0x7f1564198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f15637fe640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 0x7f1564198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f15637fe640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 0x7f1564198fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f1563fff640 1 -- 172.21.15.189:0/2920271246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 msgr2=0x7f1564198fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f1563fff640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 0x7f1564198fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f1563fff640 1 -- 172.21.15.189:0/2920271246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15641a1690 con 0x7f15641017a0 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f1563fff640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 0x7f1564198620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f15540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f15617fa640 1 -- 172.21.15.189:0/2920271246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1554035030 con 0x7f15641017a0 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15641a1860 con 0x7f15641017a0 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f15617fa640 1 -- 172.21.15.189:0/2920271246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f1554004ac0 con 0x7f15641017a0 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f15617fa640 1 -- 172.21.15.189:0/2920271246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f155403b660 con 0x7f15641017a0 2024-09-13T13:24:46.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.830+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1564196410 con 0x7f15641017a0 2024-09-13T13:24:46.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.831+0000 7f15617fa640 1 -- 172.21.15.189:0/2920271246 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7f1554042070 con 0x7f15641017a0 2024-09-13T13:24:46.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.831+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1564102a50 con 0x7f15641017a0 2024-09-13T13:24:46.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.832+0000 7f15617fa640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f154c0438f0 0x7f154c045db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:46.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.832+0000 7f15617fa640 1 -- 172.21.15.189:0/2920271246 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1554074ce0 con 0x7f15641017a0 2024-09-13T13:24:46.840 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.832+0000 7f15637fe640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f154c0438f0 0x7f154c045db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:46.840 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.832+0000 7f15637fe640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f154c0438f0 0x7f154c045db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1558009100 tx=0x7f1558008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:46.843 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:46.841+0000 7f15617fa640 1 -- 172.21.15.189:0/2920271246 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f155402f300 con 0x7f15641017a0 2024-09-13T13:24:47.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.083+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f156406b450 con 0x7f15641017a0 2024-09-13T13:24:47.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.083+0000 7f15617fa640 1 -- 172.21.15.189:0/2920271246 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1554003ad0 con 0x7f15641017a0 2024-09-13T13:24:47.086 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:24:47.086 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","modified":"2024-09-13T13:24:39.144489Z","created":"2024-09-13T13:23:08.898338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T13:24:47.086 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f154c0438f0 msgr2=0x7f154c045db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f154c0438f0 0x7f154c045db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1558009100 tx=0x7f1558008040 comp rx=0 tx=0).stop 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 msgr2=0x7f1564198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 0x7f1564198620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f15540046c0 comp rx=0 tx=0).stop 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 shutdown_connections 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f154c0438f0 0x7f154c045db0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1564198b60 0x7f1564198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 --2- 172.21.15.189:0/2920271246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15641017a0 0x7f1564198620 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 >> 172.21.15.189:0/2920271246 conn(0x7f15640fd230 msgr2=0x7f1564103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:47.088 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 shutdown_connections 2024-09-13T13:24:47.089 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:47.086+0000 7f156a7e1640 1 -- 172.21.15.189:0/2920271246 wait complete. 2024-09-13T13:24:47.222 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 systemd[1]: Starting Ceph mon.c for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:24:47.385 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/2920271246' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/2920271246' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 podman[40217]: 2024-09-13 13:24:47.190039048 +0000 UTC m=+0.016356589 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:24:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 podman[40217]: 2024-09-13 13:24:47.287954012 +0000 UTC m=+0.114271538 container create 86b07c1f07737f0e303a362e0ac36ff224d298c79176726d6ec711476a2db900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8) 2024-09-13T13:24:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 podman[40217]: 2024-09-13 13:24:47.506872696 +0000 UTC m=+0.333190222 container init 86b07c1f07737f0e303a362e0ac36ff224d298c79176726d6ec711476a2db900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T13:24:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 podman[40217]: 2024-09-13 13:24:47.511762538 +0000 UTC m=+0.338080065 container start 86b07c1f07737f0e303a362e0ac36ff224d298c79176726d6ec711476a2db900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-13T13:24:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: set uid:gid to 167:167 (ceph:ceph) 2024-09-13T13:24:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa43119222b) squid (dev), process ceph-mon, pid 7 2024-09-13T13:24:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pidfile_write: ignore empty --pid-file 2024-09-13T13:24:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: load: jerasure load: lrc 2024-09-13T13:24:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: RocksDB version: 7.9.2 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Git sha 0 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Compile date 2024-09-13 10:06:52 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: DB SUMMARY 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: DB Session ID: V81TPPCZ4J3CU0IYVRRX 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: CURRENT file: CURRENT 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.error_if_exists: 0 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.create_if_missing: 0 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.paranoid_checks: 1 2024-09-13T13:24:47.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T13:24:47.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T13:24:47.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T13:24:47.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.env: 0x55a6de50a120 2024-09-13T13:24:47.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T13:24:47.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.info_log: 0x55a6e0157700 2024-09-13T13:24:47.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.statistics: (nil) 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.use_fsync: 0 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_log_file_size: 0 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.allow_fallocate: 1 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T13:24:47.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.use_direct_reads: 0 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.db_log_dir: 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.wal_dir: 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T13:24:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.write_buffer_manager: 0x55a6e015bb80 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.unordered_write: 0 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T13:24:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.row_cache: None 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.wal_filter: None 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.two_write_queues: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.wal_compression: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.atomic_flush: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.log_readahead_size: 0 2024-09-13T13:24:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_background_jobs: 2 2024-09-13T13:24:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_background_compactions: -1 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_subcompactions: 1 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_open_files: -1 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T13:24:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_background_flushes: -1 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Compression algorithms supported: 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kZSTD supported: 0 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kXpressCompression supported: 0 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kBZip2Compression supported: 0 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kLZ4Compression supported: 1 2024-09-13T13:24:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kZlibCompression supported: 1 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: kSnappyCompression supported: 1 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.merge_operator: 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_filter: None 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T13:24:48.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a6e0157320) 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_type: 0 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_index_type: 0 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_shortening: 1 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: checksum: 4 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: no_block_cache: 0 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache: 0x55a6e014b610 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_options: 2024-09-13T13:24:48.003 INFO:journalctl@ceph.mon.c.smithi043.stdout: capacity : 536870912 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_shard_bits : 4 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: strict_capacity_limit : 0 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_compressed: (nil) 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: persistent_cache: (nil) 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size: 4096 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size_deviation: 10 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_restart_interval: 16 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_block_restart_interval: 1 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: metadata_block_size: 4096 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: partition_filters: 0 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: use_delta_encoding: 1 2024-09-13T13:24:48.004 INFO:journalctl@ceph.mon.c.smithi043.stdout: filter_policy: bloomfilter 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: whole_key_filtering: 1 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: verify_compression: 0 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: format_version: 5 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_index_compression: 1 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_align: 0 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: max_auto_readahead_size: 262144 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: prepopulate_block_cache: 0 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: initial_auto_readahead_size: 8192 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression: NoCompression 2024-09-13T13:24:48.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.num_levels: 7 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T13:24:48.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T13:24:48.007 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T13:24:48.008 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T13:24:48.009 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.inplace_update_support: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.bloom_locality: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.max_successive_merges: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T13:24:48.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.ttl: 2592000 2024-09-13T13:24:48.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T13:24:48.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.enable_blob_files: false 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.min_blob_size: 0 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T13:24:48.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 21df5561-dea6-4234-9c1b-ed799dc16858 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726233887538680, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726233887539067, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726233887, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "21df5561-dea6-4234-9c1b-ed799dc16858", "db_session_id": "V81TPPCZ4J3CU0IYVRRX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726233887539167, "job": 1, "event": "recovery_finished"} 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a6e017ae00 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: DB pointer 0x55a6e027c000 2024-09-13T13:24:48.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** DB Stats ** 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T13:24:48.014 INFO:journalctl@ceph.mon.c.smithi043.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T13:24:48.015 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache BinnedLRUCache@0x55a6e014b610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: using public_addrv [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-09-13T13:24:48.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: starting mon.c rank -1 at public addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] at bind addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(???) e0 preinit fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).mds e1 new map 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout: e1 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout: btime 2024-09-13T13:23:11:853606+0000 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout: legacy client fscid: -1 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout: No filesystems configured 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-13T13:24:48.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Added host smithi189 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2775734466' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2775734466' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"} : dispatch 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi189:172.21.15.189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi189:172.21.15.189=b;count:3 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Deploying daemon mon.b on smithi189 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/1119094768' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/4143819412' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.a calling monitor election 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:48.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.b calling monitor election 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: monmap epoch 2 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: last_changed 2024-09-13T13:24:39.144489+0000 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: min_mon_release 19 (squid) 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: election_strategy: 1 2024-09-13T13:24:48.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: fsmap 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mgrmap e14: y(active, since 37s) 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: overall HEALTH_OK 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: Deploying daemon mon.c on smithi043 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/1559803067' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:48.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/2920271246' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 ceph-mon[40255]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-13T13:24:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 bash[40217]: 86b07c1f07737f0e303a362e0ac36ff224d298c79176726d6ec711476a2db900 2024-09-13T13:24:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:47 smithi043 systemd[1]: Started Ceph mon.c for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:24:48.673 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T13:24:48.676 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T13:24:48.940 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:24:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: mon.a calling monitor election 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: mon.b calling monitor election 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: mon.c calling monitor election 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: monmap epoch 3 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: last_changed 2024-09-13T13:24:47.563513+0000 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: min_mon_release 19 (squid) 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: election_strategy: 1 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: 2: [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon.c 2024-09-13T13:24:52.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: fsmap 2024-09-13T13:24:52.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:52.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: mgrmap e14: y(active, since 45s) 2024-09-13T13:24:52.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: overall HEALTH_OK 2024-09-13T13:24:52.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.940+0000 7f73a53d7640 1 -- 172.21.15.189:0/1392019316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 msgr2=0x7f7388005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.940+0000 7f73a53d7640 1 --2- 172.21.15.189:0/1392019316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 0x7f7388005580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7390014d10 tx=0x7f739002c570 comp rx=0 tx=0).stop 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.940+0000 7f73a53d7640 1 -- 172.21.15.189:0/1392019316 shutdown_connections 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.940+0000 7f73a53d7640 1 --2- 172.21.15.189:0/1392019316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 0x7f7388005580 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.940+0000 7f73a53d7640 1 -- 172.21.15.189:0/1392019316 >> 172.21.15.189:0/1392019316 conn(0x7f73a00fd200 msgr2=0x7f73a00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.940+0000 7f73a53d7640 1 -- 172.21.15.189:0/1392019316 shutdown_connections 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.940+0000 7f73a53d7640 1 -- 172.21.15.189:0/1392019316 wait complete. 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 Processor -- start 2024-09-13T13:24:52.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 -- start start 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 0x7f73a01976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 0x7f73a019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 0x7f73a019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73a0108960 con 0x7f73a0101790 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01087e0 con 0x7f73a0197bf0 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f73a53d7640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f73a0108ae0 con 0x7f73a019d4d0 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f739e7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 0x7f73a019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.941+0000 7f739e7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 0x7f73a019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:52808/0 (socket says 172.21.15.189:52808) 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739e7fc640 1 -- 172.21.15.189:0/3127232763 learned_addr learned my addr 172.21.15.189:0/3127232763 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739effd640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 0x7f73a01976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739f7fe640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 0x7f73a019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739e7fc640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 msgr2=0x7f73a019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T13:24:52.945 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739e7fc640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 msgr2=0x7f73a019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739e7fc640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 0x7f73a019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739e7fc640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 0x7f73a019cf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739f7fe640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 msgr2=0x7f73a019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739f7fe640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 msgr2=0x7f73a019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739f7fe640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 0x7f73a019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739f7fe640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 0x7f73a019d980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739effd640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 msgr2=0x7f73a019d980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739effd640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 0x7f73a019d980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739effd640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 msgr2=0x7f73a019cf90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739effd640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 0x7f73a019cf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739effd640 1 -- 172.21.15.189:0/3127232763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a01a4110 con 0x7f73a0101790 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f739effd640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 0x7f73a01976b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7390035040 tx=0x7f7390005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:52.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f7387fff640 1 -- 172.21.15.189:0/3127232763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390041070 con 0x7f73a0101790 2024-09-13T13:24:52.947 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a01a43a0 con 0x7f73a0101790 2024-09-13T13:24:52.947 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f7387fff640 1 -- 172.21.15.189:0/3127232763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7390004470 con 0x7f73a0101790 2024-09-13T13:24:52.947 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f7387fff640 1 -- 172.21.15.189:0/3127232763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390003b60 con 0x7f73a0101790 2024-09-13T13:24:52.947 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.942+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a01a4860 con 0x7f73a0101790 2024-09-13T13:24:52.948 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.943+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a0109800 con 0x7f73a0101790 2024-09-13T13:24:52.948 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.943+0000 7f7387fff640 1 -- 172.21.15.189:0/3127232763 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7f7390003d00 con 0x7f73a0101790 2024-09-13T13:24:52.948 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.944+0000 7f7387fff640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f738c03fbd0 0x7f738c042090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:52.948 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.944+0000 7f7387fff640 1 -- 172.21.15.189:0/3127232763 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7390076460 con 0x7f73a0101790 2024-09-13T13:24:52.948 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.944+0000 7f739e7fc640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f738c03fbd0 0x7f738c042090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:52.948 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.944+0000 7f739e7fc640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f738c03fbd0 0x7f738c042090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7394009100 tx=0x7f7394008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:52.951 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:52.950+0000 7f7387fff640 1 -- 172.21.15.189:0/3127232763 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f739003b320 con 0x7f73a0101790 2024-09-13T13:24:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:24:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: mon.a calling monitor election 2024-09-13T13:24:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: mon.b calling monitor election 2024-09-13T13:24:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: mon.c calling monitor election 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: monmap epoch 3 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: last_changed 2024-09-13T13:24:47.563513+0000 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:24:52.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: min_mon_release 19 (squid) 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: election_strategy: 1 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: 2: [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon.c 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: fsmap 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: mgrmap e14: y(active, since 45s) 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: overall HEALTH_OK 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: mon.a calling monitor election 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: mon.b calling monitor election 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: mon.c calling monitor election 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: monmap epoch 3 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: last_changed 2024-09-13T13:24:47.563513+0000 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: created 2024-09-13T13:23:08.898338+0000 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: min_mon_release 19 (squid) 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: election_strategy: 1 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: 2: [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon.c 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: fsmap 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: mgrmap e14: y(active, since 45s) 2024-09-13T13:24:52.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: overall HEALTH_OK 2024-09-13T13:24:52.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:52.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:53.206 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.205+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f73a01a4ac0 con 0x7f73a0101790 2024-09-13T13:24:53.207 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.206+0000 7f7387fff640 1 -- 172.21.15.189:0/3127232763 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f7390004a50 con 0x7f73a0101790 2024-09-13T13:24:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:24:53.207 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","modified":"2024-09-13T13:24:47.563513Z","created":"2024-09-13T13:23:08.898338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-13T13:24:53.207 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-09-13T13:24:53.209 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f738c03fbd0 msgr2=0x7f738c042090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f738c03fbd0 0x7f738c042090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7394009100 tx=0x7f7394008040 comp rx=0 tx=0).stop 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 msgr2=0x7f73a01976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 0x7f73a01976b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7390035040 tx=0x7f7390005230 comp rx=0 tx=0).stop 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 shutdown_connections 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f738c03fbd0 0x7f738c042090 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73a019d4d0 0x7f73a019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0197bf0 0x7f73a019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.208+0000 7f73a53d7640 1 --2- 172.21.15.189:0/3127232763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73a0101790 0x7f73a01976b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.209+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 >> 172.21.15.189:0/3127232763 conn(0x7f73a00fd200 msgr2=0x7f73a006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.209+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 shutdown_connections 2024-09-13T13:24:53.210 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:53.209+0000 7f73a53d7640 1 -- 172.21.15.189:0/3127232763 wait complete. 2024-09-13T13:24:53.849 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:53.850 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/3127232763' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:53.850 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:53.851 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-13T13:24:53.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-13T13:24:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/3127232763' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:24:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/3127232763' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T13:24:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:24:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.561+0000 7ff60c839640 1 -- 172.21.15.43:0/2664540784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff608103670 msgr2=0x7ff608103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.561+0000 7ff60c839640 1 --2- 172.21.15.43:0/2664540784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff608103670 0x7ff608103a50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009880 tx=0x7ff5f802a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.561+0000 7ff60c839640 1 -- 172.21.15.43:0/2664540784 shutdown_connections 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.561+0000 7ff60c839640 1 --2- 172.21.15.43:0/2664540784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff608103670 0x7ff608103a50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.561+0000 7ff60c839640 1 -- 172.21.15.43:0/2664540784 >> 172.21.15.43:0/2664540784 conn(0x7ff6080fd0b0 msgr2=0x7ff6080ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.561+0000 7ff60c839640 1 -- 172.21.15.43:0/2664540784 shutdown_connections 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.561+0000 7ff60c839640 1 -- 172.21.15.43:0/2664540784 wait complete. 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 Processor -- start 2024-09-13T13:24:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 -- start start 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff608103670 0x7ff608197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff608197cb0 0x7ff60819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60819d590 0x7ff60819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60659c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff608103670 0x7ff608197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60659c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff608103670 0x7ff608197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54836/0 (socket says 172.21.15.43:54836) 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60659c640 1 -- 172.21.15.43:0/2854672497 learned_addr learned my addr 172.21.15.43:0/2854672497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff60810bee0 con 0x7ff60819d590 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff60810bd60 con 0x7ff608197cb0 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7ff60810c060 con 0x7ff608103670 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.562+0000 7ff605d9b640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff608197cb0 0x7ff60819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff606d9d640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60819d590 0x7ff60819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff605d9b640 1 -- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff608103670 msgr2=0x7ff608197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff605d9b640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff608103670 0x7ff608197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff605d9b640 1 -- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60819d590 msgr2=0x7ff60819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff605d9b640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60819d590 0x7ff60819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff605d9b640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff608101200 con 0x7ff608197cb0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff605d9b640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff608197cb0 0x7ff60819d050 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b450 tx=0x7ff5fc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff5f77fe640 1 -- 172.21.15.43:0/2854672497 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc004230 con 0x7ff608197cb0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff5f77fe640 1 -- 172.21.15.43:0/2854672497 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ff5fc0043d0 con 0x7ff608197cb0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff5f77fe640 1 -- 172.21.15.43:0/2854672497 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0039d0 con 0x7ff608197cb0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.563+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6081014f0 con 0x7ff608197cb0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.564+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff608101a60 con 0x7ff608197cb0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.564+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60806b530 con 0x7ff608197cb0 2024-09-13T13:24:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.565+0000 7ff5f77fe640 1 -- 172.21.15.43:0/2854672497 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7ff5fc00f050 con 0x7ff608197cb0 2024-09-13T13:24:54.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.565+0000 7ff5f77fe640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff5f003fbd0 0x7ff5f0042090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:54.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.565+0000 7ff5f77fe640 1 -- 172.21.15.43:0/2854672497 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff5fc053ad0 con 0x7ff608197cb0 2024-09-13T13:24:54.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.565+0000 7ff60659c640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff5f003fbd0 0x7ff5f0042090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:54.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.565+0000 7ff60659c640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff5f003fbd0 0x7ff5f0042090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005ec0 tx=0x7ff5f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:54.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.570+0000 7ff5f77fe640 1 -- 172.21.15.43:0/2854672497 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff5fc00d040 con 0x7ff608197cb0 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:54.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:54.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:54.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.768+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff6081988a0 con 0x7ff608197cb0 2024-09-13T13:24:54.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.769+0000 7ff5f77fe640 1 -- 172.21.15.43:0/2854672497 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7ff5fc011870 con 0x7ff608197cb0 2024-09-13T13:24:54.770 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:54.770 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-13T13:24:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:24:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-09-13T13:24:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff5f003fbd0 msgr2=0x7ff5f0042090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff5f003fbd0 0x7ff5f0042090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005ec0 tx=0x7ff5f8005a20 comp rx=0 tx=0).stop 2024-09-13T13:24:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff608197cb0 msgr2=0x7ff60819d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff608197cb0 0x7ff60819d050 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b450 tx=0x7ff5fc00b910 comp rx=0 tx=0).stop 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 shutdown_connections 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7ff5f003fbd0 0x7ff5f0042090 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60819d590 0x7ff60819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff608197cb0 0x7ff60819d050 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 --2- 172.21.15.43:0/2854672497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff608103670 0x7ff608197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 >> 172.21.15.43:0/2854672497 conn(0x7ff6080fd0b0 msgr2=0x7ff608106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 shutdown_connections 2024-09-13T13:24:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:24:54.771+0000 7ff60c839640 1 -- 172.21.15.43:0/2854672497 wait complete. 2024-09-13T13:24:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:55.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:55.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:55.363 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-13T13:24:55.363 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:24:55.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T13:24:55.390 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:24:55.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:24:55.456 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:24:55.456 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T13:24:55.488 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:24:55.488 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:24:55.556 INFO:tasks.cephadm:Adding mgr.y on smithi043 2024-09-13T13:24:55.556 INFO:tasks.cephadm:Adding mgr.x on smithi189 2024-09-13T13:24:55.557 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi043=y;smithi189=x' 2024-09-13T13:24:55.842 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:24:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:55 smithi189 ceph-mon[32912]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-13T13:24:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:55 smithi189 ceph-mon[32912]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:24:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:55 smithi189 ceph-mon[32912]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:55 smithi189 ceph-mon[32912]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:55 smithi189 ceph-mon[32912]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T13:24:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:55 smithi189 ceph-mon[32912]: Reconfiguring daemon mon.a on smithi043 2024-09-13T13:24:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2854672497' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[30106]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[30106]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[30106]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[30106]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[30106]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[30106]: Reconfiguring daemon mon.a on smithi043 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2854672497' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[40255]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-13T13:24:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[40255]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:24:55.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[40255]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:55.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[40255]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:24:55.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[40255]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T13:24:55.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[40255]: Reconfiguring daemon mon.a on smithi043 2024-09-13T13:24:55.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2854672497' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.248+0000 7f3586840640 1 -- 172.21.15.189:0/2682559588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 msgr2=0x7f3580105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.248+0000 7f3586840640 1 --2- 172.21.15.189:0/2682559588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 0x7f3580105110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3570009880 tx=0x7f357002a5d0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.248+0000 7f3586840640 1 -- 172.21.15.189:0/2682559588 shutdown_connections 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.248+0000 7f3586840640 1 --2- 172.21.15.189:0/2682559588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 0x7f3580105110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.248+0000 7f3586840640 1 -- 172.21.15.189:0/2682559588 >> 172.21.15.189:0/2682559588 conn(0x7f35800fcd00 msgr2=0x7f35800ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.248+0000 7f3586840640 1 -- 172.21.15.189:0/2682559588 shutdown_connections 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.248+0000 7f3586840640 1 -- 172.21.15.189:0/2682559588 wait complete. 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 Processor -- start 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 -- start start 2024-09-13T13:24:56.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 0x7f3580197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35801979b0 0x7f3580198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3580198bd0 0x7f3580199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35801079f0 con 0x7f358006bb70 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3580107870 con 0x7f35801979b0 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3586840640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f3580107b70 con 0x7f3580198bd0 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35801979b0 0x7f3580198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35801979b0 0x7f3580198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:52814/0 (socket says 172.21.15.189:52814) 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f357f7fe640 1 -- 172.21.15.189:0/1140957262 learned_addr learned my addr 172.21.15.189:0/1140957262 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f357ffff640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 0x7f3580197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.249+0000 7f3584ddd640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3580198bd0 0x7f3580199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357f7fe640 1 -- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3580198bd0 msgr2=0x7f3580199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357f7fe640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3580198bd0 0x7f3580199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357f7fe640 1 -- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 msgr2=0x7f3580197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:56.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357f7fe640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 0x7f3580197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357f7fe640 1 -- 172.21.15.189:0/1140957262 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358019f7b0 con 0x7f35801979b0 2024-09-13T13:24:56.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357ffff640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 0x7f3580197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:24:56.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357f7fe640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35801979b0 0x7f3580198690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f357400b520 tx=0x7f357400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:56.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357d7fa640 1 -- 172.21.15.189:0/1140957262 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574004230 con 0x7f35801979b0 2024-09-13T13:24:56.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357d7fa640 1 -- 172.21.15.189:0/1140957262 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f35740043d0 con 0x7f35801979b0 2024-09-13T13:24:56.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358019faa0 con 0x7f35801979b0 2024-09-13T13:24:56.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f357d7fa640 1 -- 172.21.15.189:0/1140957262 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574003a70 con 0x7f35801979b0 2024-09-13T13:24:56.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.250+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358019ffe0 con 0x7f35801979b0 2024-09-13T13:24:56.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.251+0000 7f357d7fa640 1 -- 172.21.15.189:0/1140957262 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7f357400e070 con 0x7f35801979b0 2024-09-13T13:24:56.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.251+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3580103180 con 0x7f35801979b0 2024-09-13T13:24:56.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.251+0000 7f357d7fa640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f356803fbd0 0x7f3568042090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:24:56.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.251+0000 7f357d7fa640 1 -- 172.21.15.189:0/1140957262 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f35740538b0 con 0x7f35801979b0 2024-09-13T13:24:56.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.252+0000 7f357ffff640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f356803fbd0 0x7f3568042090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:24:56.260 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.252+0000 7f357ffff640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f356803fbd0 0x7f3568042090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3570000c00 tx=0x7f3570009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:24:56.260 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.256+0000 7f357d7fa640 1 -- 172.21.15.189:0/1140957262 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3574014020 con 0x7f35801979b0 2024-09-13T13:24:56.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.444+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi189=x", "target": ["mon-mgr", ""]}) -- 0x7f3580063590 con 0x7f356803fbd0 2024-09-13T13:24:56.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.450+0000 7f357d7fa640 1 -- 172.21.15.189:0/1140957262 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3580063590 con 0x7f356803fbd0 2024-09-13T13:24:56.451 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-09-13T13:24:56.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f356803fbd0 msgr2=0x7f3568042090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:56.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f356803fbd0 0x7f3568042090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3570000c00 tx=0x7f3570009600 comp rx=0 tx=0).stop 2024-09-13T13:24:56.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35801979b0 msgr2=0x7f3580198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:24:56.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35801979b0 0x7f3580198690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f357400b520 tx=0x7f357400b9e0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 shutdown_connections 2024-09-13T13:24:56.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f356803fbd0 0x7f3568042090 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3580198bd0 0x7f3580199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35801979b0 0x7f3580198690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 --2- 172.21.15.189:0/1140957262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358006bb70 0x7f3580197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:24:56.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 >> 172.21.15.189:0/1140957262 conn(0x7f35800fcd00 msgr2=0x7f35800fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:24:56.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 shutdown_connections 2024-09-13T13:24:56.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:24:56.452+0000 7f3586840640 1 -- 172.21.15.189:0/1140957262 wait complete. 2024-09-13T13:24:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[40255]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[30106]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:56.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:57.013 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:56 smithi189 ceph-mon[32912]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:57.013 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:57.013 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:57.013 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:57.013 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:57.013 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:57.013 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:57.040 DEBUG:teuthology.orchestra.run.smithi189:mgr.x> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.x.service 2024-09-13T13:24:57.043 DEBUG:tasks.cephadm:set 0 configs 2024-09-13T13:24:57.043 INFO:tasks.cephadm:Deploying OSDs... 2024-09-13T13:24:57.043 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:24:57.043 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T13:24:57.060 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T13:24:57.060 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-13T13:24:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 959 Links: 1 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 13:24:47.012211786 +0000 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 13:20:11.541544022 +0000 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 13:20:11.541544022 +0000 2024-09-13T13:24:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-13 13:20:11.541544022 +0000 2024-09-13T13:24:57.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T13:24:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T13:24:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T13:24:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000106205 s, 4.8 MB/s 2024-09-13T13:24:57.182 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T13:24:57.240 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-13T13:24:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T13:24:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:57.296 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 958 Links: 1 2024-09-13T13:24:57.296 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:57.296 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:57.297 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 13:24:47.012211786 +0000 2024-09-13T13:24:57.297 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 13:20:11.541544022 +0000 2024-09-13T13:24:57.297 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 13:20:11.541544022 +0000 2024-09-13T13:24:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-13 13:20:11.541544022 +0000 2024-09-13T13:24:57.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T13:24:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T13:24:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T13:24:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000105224 s, 4.9 MB/s 2024-09-13T13:24:57.362 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T13:24:57.419 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-13T13:24:57.476 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 949 Links: 1 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 13:24:47.013211774 +0000 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 13:20:11.540544012 +0000 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 13:20:11.540544012 +0000 2024-09-13T13:24:57.477 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-13 13:20:11.540544012 +0000 2024-09-13T13:24:57.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T13:24:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T13:24:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T13:24:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000107321 s, 4.8 MB/s 2024-09-13T13:24:57.543 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T13:24:57.600 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-13T13:24:57.656 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T13:24:57.656 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:57.656 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-09-13T13:24:57.656 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:57.656 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:57.656 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 13:24:47.013211774 +0000 2024-09-13T13:24:57.657 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 13:20:11.540544012 +0000 2024-09-13T13:24:57.657 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 13:20:11.540544012 +0000 2024-09-13T13:24:57.657 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-13 13:20:11.540544012 +0000 2024-09-13T13:24:57.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T13:24:57.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T13:24:57.722 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T13:24:57.722 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000109047 s, 4.7 MB/s 2024-09-13T13:24:57.723 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T13:24:57.782 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:24:57.782 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T13:24:57.799 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T13:24:57.799 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-13T13:24:57.855 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T13:24:57.855 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:57.855 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-13T13:24:57.855 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:57.855 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:57.855 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-13 13:24:37.666093638 +0000 2024-09-13T13:24:57.855 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-13 13:20:11.496306147 +0000 2024-09-13T13:24:57.856 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-13 13:20:11.496306147 +0000 2024-09-13T13:24:57.856 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-13 13:20:11.496306147 +0000 2024-09-13T13:24:57.856 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T13:24:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-13T13:24:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-13T13:24:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000232436 s, 2.2 MB/s 2024-09-13T13:24:57.920 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T13:24:57.978 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-13T13:24:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[30106]: Reconfiguring mon.c (monmap changed)... 2024-09-13T13:24:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[30106]: Reconfiguring daemon mon.c on smithi043 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[30106]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[30106]: Saving service mgr spec with placement smithi043=y;smithi189=x;count:2 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[40255]: Reconfiguring mon.c (monmap changed)... 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[40255]: Reconfiguring daemon mon.c on smithi043 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[40255]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[40255]: Saving service mgr spec with placement smithi043=y;smithi189=x;count:2 2024-09-13T13:24:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-13 13:24:37.666093638 +0000 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-13 13:20:11.498306218 +0000 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-13 13:20:11.498306218 +0000 2024-09-13T13:24:58.034 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-13 13:20:11.498306218 +0000 2024-09-13T13:24:58.035 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T13:24:58.095 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:57 smithi189 ceph-mon[32912]: Reconfiguring mon.c (monmap changed)... 2024-09-13T13:24:58.095 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:57 smithi189 ceph-mon[32912]: Reconfiguring daemon mon.c on smithi043 2024-09-13T13:24:58.095 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:57 smithi189 ceph-mon[32912]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:24:58.095 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:57 smithi189 ceph-mon[32912]: Saving service mgr spec with placement smithi043=y;smithi189=x;count:2 2024-09-13T13:24:58.095 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:58.097 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-13T13:24:58.097 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-13T13:24:58.097 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000310193 s, 1.7 MB/s 2024-09-13T13:24:58.099 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T13:24:58.155 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-13T13:24:58.211 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-13 13:24:37.667093619 +0000 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-13 13:20:11.498306218 +0000 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-13 13:20:11.498306218 +0000 2024-09-13T13:24:58.212 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-13 13:20:11.498306218 +0000 2024-09-13T13:24:58.212 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T13:24:58.276 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-13T13:24:58.276 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-13T13:24:58.276 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000681133 s, 752 kB/s 2024-09-13T13:24:58.278 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T13:24:58.334 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-13T13:24:58.389 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T13:24:58.389 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T13:24:58.389 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-13T13:24:58.389 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T13:24:58.389 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T13:24:58.389 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-13 13:24:37.667093619 +0000 2024-09-13T13:24:58.390 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-13 13:20:11.497306182 +0000 2024-09-13T13:24:58.390 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-13 13:20:11.497306182 +0000 2024-09-13T13:24:58.390 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-13 13:20:11.497306182 +0000 2024-09-13T13:24:58.390 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T13:24:58.454 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-13T13:24:58.454 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-13T13:24:58.454 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000189516 s, 2.7 MB/s 2024-09-13T13:24:58.455 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T13:24:58.511 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-09-13T13:24:58.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-13T13:24:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:24:58.874 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: Reconfiguring mon.b (monmap changed)... 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[30106]: Reconfiguring daemon mon.b on smithi189 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:58.875 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:58.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:58.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: Reconfiguring mon.b (monmap changed)... 2024-09-13T13:24:58.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:58.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:58.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:58.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:24:58 smithi043 ceph-mon[40255]: Reconfiguring daemon mon.b on smithi189 2024-09-13T13:24:59.069 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:24:59.070 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:59.070 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:24:59.070 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: Reconfiguring mon.b (monmap changed)... 2024-09-13T13:24:59.070 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T13:24:59.070 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T13:24:59.070 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:24:59.070 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:24:58 smithi189 ceph-mon[32912]: Reconfiguring daemon mon.b on smithi189 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: Deploying daemon mgr.x on smithi189 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[30106]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T13:25:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: Deploying daemon mgr.x on smithi189 2024-09-13T13:25:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:00 smithi043 ceph-mon[40255]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T13:25:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T13:25:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T13:25:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: Deploying daemon mgr.x on smithi189 2024-09-13T13:25:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:00 smithi189 ceph-mon[32912]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:01.061 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:00 smithi189 systemd[1]: Starting Ceph mgr.x for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:25:01.355 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:01 smithi189 podman[34549]: 2024-09-13 13:25:01.060682961 +0000 UTC m=+0.108343695 container create 6ff729135afe75df9efee91ef0a142a7bb530652632ba4dca023dcd8406bd378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-13T13:25:01.355 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:01 smithi189 podman[34549]: 2024-09-13 13:25:00.968104852 +0000 UTC m=+0.015765600 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:25:01.355 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:01 smithi189 podman[34549]: 2024-09-13 13:25:01.279644061 +0000 UTC m=+0.327304792 container init 6ff729135afe75df9efee91ef0a142a7bb530652632ba4dca023dcd8406bd378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-13T13:25:01.355 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:01 smithi189 podman[34549]: 2024-09-13 13:25:01.284502264 +0000 UTC m=+0.332162995 container start 6ff729135afe75df9efee91ef0a142a7bb530652632ba4dca023dcd8406bd378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-13T13:25:01.355 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:01 smithi189 ceph-mgr[34588]: -- 172.21.15.189:0/2936744913 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6a1dc50e0 con 0x55b6a1da2800 2024-09-13T13:25:01.355 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:01 smithi189 bash[34549]: 6ff729135afe75df9efee91ef0a142a7bb530652632ba4dca023dcd8406bd378 2024-09-13T13:25:01.399 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:25:01.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-09-13T13:25:01.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:25:01 smithi189 systemd[1]: Started Ceph mgr.x for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:25:01.687 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:25:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 -- 172.21.15.43:0/2933141381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520102a10 msgr2=0x7fc520102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 --2- 172.21.15.43:0/2933141381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520102a10 0x7fc520102e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc510009930 tx=0x7fc51002a420 comp rx=0 tx=0).stop 2024-09-13T13:25:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 -- 172.21.15.43:0/2933141381 shutdown_connections 2024-09-13T13:25:02.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 --2- 172.21.15.43:0/2933141381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc520104530 0x7fc52010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 --2- 172.21.15.43:0/2933141381 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520103b70 0x7fc520103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 --2- 172.21.15.43:0/2933141381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520102a10 0x7fc520102e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 -- 172.21.15.43:0/2933141381 >> 172.21.15.43:0/2933141381 conn(0x7fc5200fe2c0 msgr2=0x7fc520100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:25:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 -- 172.21.15.43:0/2933141381 shutdown_connections 2024-09-13T13:25:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.159+0000 7fc525176640 1 -- 172.21.15.43:0/2933141381 wait complete. 2024-09-13T13:25:02.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 Processor -- start 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 -- start start 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520102a10 0x7fc520198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520103b70 0x7fc5201989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc520104530 0x7fc520199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc52010db00 con 0x7fc520103b70 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc52010d980 con 0x7fc520104530 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc525176640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fc52010dc80 con 0x7fc520102a10 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc51e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520103b70 0x7fc5201989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc51ed9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520102a10 0x7fc520198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc51e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520103b70 0x7fc5201989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40752/0 (socket says 172.21.15.43:40752) 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc51ed9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520102a10 0x7fc520198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:34398/0 (socket says 172.21.15.43:34398) 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc51e59c640 1 -- 172.21.15.43:0/4080240155 learned_addr learned my addr 172.21.15.43:0/4080240155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.160+0000 7fc51f59e640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc520104530 0x7fc520199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc51ed9d640 1 -- 172.21.15.43:0/4080240155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc520104530 msgr2=0x7fc520199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc51ed9d640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc520104530 0x7fc520199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc51ed9d640 1 -- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520103b70 msgr2=0x7fc5201989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc51ed9d640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520103b70 0x7fc5201989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc51ed9d640 1 -- 172.21.15.43:0/4080240155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc520199eb0 con 0x7fc520102a10 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc51e59c640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520103b70 0x7fc5201989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc51ed9d640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520102a10 0x7fc520198380 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc510009900 tx=0x7fc510004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:25:02.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51003f070 con 0x7fc520102a10 2024-09-13T13:25:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc510005450 con 0x7fc520102a10 2024-09-13T13:25:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52019a140 con 0x7fc520102a10 2024-09-13T13:25:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510035030 con 0x7fc520102a10 2024-09-13T13:25:02.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.161+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5201a06b0 con 0x7fc520102a10 2024-09-13T13:25:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.162+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 14) ==== 51821+0+0 (secure 0 0 0) 0x7fc510042070 con 0x7fc520102a10 2024-09-13T13:25:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.162+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc520102e10 con 0x7fc520102a10 2024-09-13T13:25:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.163+0000 7fc507fff640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc50803f980 0x7fc508041e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.163+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc510074b50 con 0x7fc520102a10 2024-09-13T13:25:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.163+0000 7fc51e59c640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc50803f980 0x7fc508041e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.163+0000 7fc51e59c640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc50803f980 0x7fc508041e40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc514011500 tx=0x7fc514005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:25:02.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.167+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc51002f300 con 0x7fc520102a10 2024-09-13T13:25:02.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:02.363+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc520063590 con 0x7fc50803f980 2024-09-13T13:25:02.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:02.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:02.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:25:02.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:25:02.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:25:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:25:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:03.728 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: Reconfiguring mgr.y (unknown last config time)... 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:03.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[30106]: Reconfiguring daemon mgr.y on smithi043 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: Reconfiguring mgr.y (unknown last config time)... 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T13:25:03.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:03.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:03 smithi043 ceph-mon[40255]: Reconfiguring daemon mgr.y on smithi043 2024-09-13T13:25:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:25:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: Reconfiguring mgr.y (unknown last config time)... 2024-09-13T13:25:03.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T13:25:03.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:03.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T13:25:03.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:03.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:03 smithi189 ceph-mon[32912]: Reconfiguring daemon mgr.y on smithi043 2024-09-13T13:25:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:04 smithi189 ceph-mon[32912]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:04.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:04 smithi043 ceph-mon[30106]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:04.903 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:04 smithi043 ceph-mon[40255]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:06.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[40255]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:06.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:06.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:05 smithi043 ceph-mon[30106]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:05 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:05 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:05 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:05 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:05 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:05 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:05 smithi189 ceph-mon[32912]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:07.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:07.030+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mgrmap(e 15) ==== 103023+0+0 (secure 0 0 0) 0x7fc510031a60 con 0x7fc520102a10 2024-09-13T13:25:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2099077043' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"} : dispatch 2024-09-13T13:25:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"} : dispatch 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"}]': finished 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: Standby manager daemon x started 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T13:25:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/942986942' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:25:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2099077043' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"} : dispatch 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"} : dispatch 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"}]': finished 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: Standby manager daemon x started 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T13:25:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/942986942' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2099077043' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"} : dispatch 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"} : dispatch 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bd16e07-e057-4c72-83d3-17490e876e64"}]': finished 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: Standby manager daemon x started 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T13:25:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/4037980688' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T13:25:07.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/942986942' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:25:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:08 smithi189 ceph-mon[32912]: mgrmap e15: y(active, since 59s), standbys: x 2024-09-13T13:25:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:08 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T13:25:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:08 smithi189 ceph-mon[32912]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:08 smithi043 ceph-mon[30106]: mgrmap e15: y(active, since 59s), standbys: x 2024-09-13T13:25:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:08 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T13:25:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:08 smithi043 ceph-mon[30106]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:08 smithi043 ceph-mon[40255]: mgrmap e15: y(active, since 59s), standbys: x 2024-09-13T13:25:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:08 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T13:25:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:08 smithi043 ceph-mon[40255]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:10 smithi043 ceph-mon[30106]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:10 smithi043 ceph-mon[40255]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:10 smithi189 ceph-mon[32912]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:12.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:12 smithi043 ceph-mon[30106]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:12.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:12 smithi043 ceph-mon[40255]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:12.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:12 smithi189 ceph-mon[32912]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:14.356 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[40255]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:14.357 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-13T13:25:14.357 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:14.357 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[40255]: Deploying daemon osd.0 on smithi043 2024-09-13T13:25:14.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[30106]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:14.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-13T13:25:14.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:14.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:14 smithi043 ceph-mon[30106]: Deploying daemon osd.0 on smithi043 2024-09-13T13:25:14.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:14 smithi189 ceph-mon[32912]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:14.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:14 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-13T13:25:14.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:14 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:14.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:14 smithi189 ceph-mon[32912]: Deploying daemon osd.0 on smithi043 2024-09-13T13:25:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:16 smithi043 ceph-mon[30106]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:16 smithi043 ceph-mon[40255]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:16.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:16 smithi189 ceph-mon[32912]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:18 smithi043 ceph-mon[30106]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:18 smithi043 ceph-mon[40255]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:18 smithi189 ceph-mon[32912]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:20.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:20 smithi043 ceph-mon[30106]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:20 smithi043 ceph-mon[40255]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:20.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:20 smithi189 ceph-mon[32912]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:21 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:21 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:21 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:21 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:21 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:21.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:21 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:21.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:21 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:21.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:21 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:21.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:21 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:22.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:22 smithi043 ceph-mon[30106]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:22.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:22 smithi043 ceph-mon[40255]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:22 smithi189 ceph-mon[32912]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[30106]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[40255]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:24.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:24.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:24.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:24.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:24.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:24 smithi189 ceph-mon[32912]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:25 smithi043 ceph-mon[40255]: from='osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T13:25:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:25 smithi043 ceph-mon[40255]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T13:25:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:25 smithi043 ceph-mon[30106]: from='osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T13:25:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:25 smithi043 ceph-mon[30106]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T13:25:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.044+0000 7fc507fff640 1 -- 172.21.15.43:0/4080240155 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc520063590 con 0x7fc50803f980 2024-09-13T13:25:26.045 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.047+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc50803f980 msgr2=0x7fc508041e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.047+0000 7fc525176640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc50803f980 0x7fc508041e40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc514011500 tx=0x7fc514005f90 comp rx=0 tx=0).stop 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.047+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520102a10 msgr2=0x7fc520198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.047+0000 7fc525176640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520102a10 0x7fc520198380 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc510009900 tx=0x7fc510004040 comp rx=0 tx=0).stop 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 shutdown_connections 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc50803f980 0x7fc508041e40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc520104530 0x7fc520199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc520103b70 0x7fc5201989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 --2- 172.21.15.43:0/4080240155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc520102a10 0x7fc520198380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 >> 172.21.15.43:0/4080240155 conn(0x7fc5200fe2c0 msgr2=0x7fc5200fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:25:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 shutdown_connections 2024-09-13T13:25:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:26.048+0000 7fc525176640 1 -- 172.21.15.43:0/4080240155 wait complete. 2024-09-13T13:25:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:25 smithi189 ceph-mon[32912]: from='osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T13:25:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:25 smithi189 ceph-mon[32912]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T13:25:26.694 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.0.service 2024-09-13T13:25:26.698 INFO:tasks.cephadm:Deploying osd.1 on smithi043 with /dev/vg_nvme/lv_3... 2024-09-13T13:25:26.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-13T13:25:26.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:26.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T13:25:26.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T13:25:26.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: from='osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:26.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:26.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:26.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:26.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:26.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:26.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: from='osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:26.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T13:25:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T13:25:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: from='osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:27.242 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0[43902]: 2024-09-13T13:25:26.993+0000 7fddbb2a9640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T13:25:27.242 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:25:26 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0[43902]: 2024-09-13T13:25:26.997+0000 7fddb28b4640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T13:25:27.465 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:25:27.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[30106]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:25:27.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[30106]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T13:25:27.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:27.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:27.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[30106]: from='osd.0 ' entity='osd.0' 2024-09-13T13:25:27.864 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[40255]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:25:27.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[40255]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T13:25:27.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:27.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:27.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:27 smithi043 ceph-mon[40255]: from='osd.0 ' entity='osd.0' 2024-09-13T13:25:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:27 smithi189 ceph-mon[32912]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:25:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:27 smithi189 ceph-mon[32912]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T13:25:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:27 smithi189 ceph-mon[32912]: from='osd.0 ' entity='osd.0' 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[30106]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[30106]: osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] boot 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[30106]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:25:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:25:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[40255]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[40255]: osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] boot 2024-09-13T13:25:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[40255]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T13:25:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:28 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:28 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:25:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:28 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:25:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:28 smithi189 ceph-mon[32912]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T13:25:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:28 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:28 smithi189 ceph-mon[32912]: osd.0 [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] boot 2024-09-13T13:25:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:28 smithi189 ceph-mon[32912]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T13:25:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:28 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:25:30.214 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:30 smithi043 ceph-mon[30106]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T13:25:30.215 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:30 smithi043 ceph-mon[30106]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:30.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:30 smithi043 ceph-mon[40255]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T13:25:30.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:30 smithi043 ceph-mon[40255]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:30 smithi189 ceph-mon[32912]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T13:25:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:30 smithi189 ceph-mon[32912]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:31.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi043 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi043 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:31.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi043 2024-09-13T13:25:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:25:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-13T13:25:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:31.917 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:25:31.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-09-13T13:25:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:25:32.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:32 smithi043 ceph-mon[30106]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:32.591 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:32 smithi043 ceph-mon[40255]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:32.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:32 smithi189 ceph-mon[32912]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:32.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 -- 172.21.15.43:0/2579039458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3204075b20 msgr2=0x7f3204075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2579039458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3204075b20 0x7f3204075fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009900 tx=0x7f31f802a410 comp rx=0 tx=0).stop 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 -- 172.21.15.43:0/2579039458 shutdown_connections 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2579039458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f320410a940 0x7f320410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2579039458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3204075b20 0x7f3204075fa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2579039458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204076f80 0x7f32040755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 -- 172.21.15.43:0/2579039458 >> 172.21.15.43:0/2579039458 conn(0x7f32040fe2c0 msgr2=0x7f3204100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 -- 172.21.15.43:0/2579039458 shutdown_connections 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.679+0000 7f320a6ac640 1 -- 172.21.15.43:0/2579039458 wait complete. 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 Processor -- start 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 -- start start 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204075b20 0x7f320419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3204076f80 0x7f320419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f320410a940 0x7f320419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f320410fd40 con 0x7f3204075b20 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f320410fbc0 con 0x7f3204076f80 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.680+0000 7f320a6ac640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f320410fec0 con 0x7f320410a940 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204075b20 0x7f320419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3208c49640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f320410a940 0x7f320419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204075b20 0x7f320419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58008/0 (socket says 172.21.15.43:58008) 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 -- 172.21.15.43:0/2323925058 learned_addr learned my addr 172.21.15.43:0/2323925058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f32037fe640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3204076f80 0x7f320419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 -- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f320410a940 msgr2=0x7f320419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f320410a940 0x7f320419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 -- 172.21.15.43:0/2323925058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3204076f80 msgr2=0x7f320419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3204076f80 0x7f320419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 -- 172.21.15.43:0/2323925058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419c080 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3208c49640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f320410a940 0x7f320419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f3203fff640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204075b20 0x7f320419a5c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f31f400ea60 tx=0x7f31f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f32017fa640 1 -- 172.21.15.43:0/2323925058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f400cde0 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f32017fa640 1 -- 172.21.15.43:0/2323925058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f31f4004470 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320419c370 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f32017fa640 1 -- 172.21.15.43:0/2323925058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4010430 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.681+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a2a50 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.682+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3204075fa0 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.683+0000 7f32017fa640 1 -- 172.21.15.43:0/2323925058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) ==== 103023+0+0 (secure 0 0 0) 0x7f31f4003f60 con 0x7f3204075b20 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.684+0000 7f32017fa640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f31ec07ac90 0x7f31ec07d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:25:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.684+0000 7f32017fa640 1 -- 172.21.15.43:0/2323925058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f31f409cee0 con 0x7f3204075b20 2024-09-13T13:25:32.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.684+0000 7f32037fe640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f31ec07ac90 0x7f31ec07d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:25:32.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.684+0000 7f32037fe640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f31ec07ac90 0x7f31ec07d150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31f80057c0 tx=0x7f31f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:25:32.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.688+0000 7f32017fa640 1 -- 172.21.15.43:0/2323925058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f31f4065520 con 0x7f3204075b20 2024-09-13T13:25:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:32.882+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f320402e930 con 0x7f31ec07ac90 2024-09-13T13:25:33.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:25:33.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:25:33.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:33 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:33.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:33 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:25:33.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:33 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:25:33.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:33 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:33 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:25:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:33 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:25:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:33 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:34.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:34 smithi043 ceph-mon[40255]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:25:34.543 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:34 smithi043 ceph-mon[40255]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:34.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:34 smithi043 ceph-mon[30106]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:25:34.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:34 smithi043 ceph-mon[30106]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:34 smithi189 ceph-mon[32912]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:25:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:34 smithi189 ceph-mon[32912]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:35.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1138876462' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"} : dispatch 2024-09-13T13:25:35.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"} : dispatch 2024-09-13T13:25:35.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"}]': finished 2024-09-13T13:25:35.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[30106]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T13:25:35.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:35.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1138876462' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"} : dispatch 2024-09-13T13:25:35.601 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"} : dispatch 2024-09-13T13:25:35.601 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"}]': finished 2024-09-13T13:25:35.601 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[40255]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T13:25:35.601 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:35 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1138876462' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"} : dispatch 2024-09-13T13:25:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"} : dispatch 2024-09-13T13:25:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950ef5d7-8755-4a89-b1b3-953dd7f3bd0f"}]': finished 2024-09-13T13:25:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:35 smithi189 ceph-mon[32912]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T13:25:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:35 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:36 smithi189 ceph-mon[32912]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3792821533' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:25:36.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:36 smithi043 ceph-mon[30106]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3792821533' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:25:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:36 smithi043 ceph-mon[40255]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3792821533' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:25:38.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:38 smithi189 ceph-mon[32912]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:38.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:38 smithi043 ceph-mon[30106]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:38 smithi043 ceph-mon[40255]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:40 smithi043 ceph-mon[30106]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:40 smithi043 ceph-mon[40255]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:40.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:40 smithi189 ceph-mon[32912]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:42.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:42 smithi043 ceph-mon[30106]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:42.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:42 smithi043 ceph-mon[40255]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:42.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:42 smithi189 ceph-mon[32912]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:43 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-13T13:25:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:43 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:43 smithi189 ceph-mon[32912]: Deploying daemon osd.1 on smithi043 2024-09-13T13:25:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:43 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-13T13:25:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:43 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:43 smithi043 ceph-mon[30106]: Deploying daemon osd.1 on smithi043 2024-09-13T13:25:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:43 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-13T13:25:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:43 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:43 smithi043 ceph-mon[40255]: Deploying daemon osd.1 on smithi043 2024-09-13T13:25:44.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:44 smithi043 ceph-mon[30106]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:44.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:44 smithi043 ceph-mon[40255]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:44.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:44 smithi189 ceph-mon[32912]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:46 smithi189 ceph-mon[32912]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:46 smithi043 ceph-mon[40255]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:46 smithi043 ceph-mon[30106]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:48 smithi189 ceph-mon[32912]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:48 smithi043 ceph-mon[30106]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:48 smithi043 ceph-mon[40255]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:50.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[30106]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:50.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:50.598 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:50.598 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:50.598 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[40255]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:50.598 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:50.598 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:50.598 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:50 smithi189 ceph-mon[32912]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:52 smithi189 ceph-mon[32912]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:52 smithi043 ceph-mon[30106]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:52 smithi043 ceph-mon[40255]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:25:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:25:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:54 smithi189 ceph-mon[32912]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:54 smithi189 ceph-mon[32912]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-13T13:25:54.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:54 smithi043 ceph-mon[30106]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:54.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:54 smithi043 ceph-mon[30106]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-13T13:25:54.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:54 smithi043 ceph-mon[40255]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:54.977 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:54 smithi043 ceph-mon[40255]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-13T13:25:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.297+0000 7f32017fa640 1 -- 172.21.15.43:0/2323925058 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f320402e930 con 0x7f31ec07ac90 2024-09-13T13:25:55.298 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 1 on host 'smithi043' 2024-09-13T13:25:55.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f31ec07ac90 msgr2=0x7f31ec07d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:55.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f31ec07ac90 0x7f31ec07d150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31f80057c0 tx=0x7f31f8000bc0 comp rx=0 tx=0).stop 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204075b20 msgr2=0x7f320419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204075b20 0x7f320419a5c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f31f400ea60 tx=0x7f31f400ef20 comp rx=0 tx=0).stop 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 shutdown_connections 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f31ec07ac90 0x7f31ec07d150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f320410a940 0x7f320419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3204076f80 0x7f320419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 --2- 172.21.15.43:0/2323925058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204075b20 0x7f320419a5c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 >> 172.21.15.43:0/2323925058 conn(0x7f32040fe2c0 msgr2=0x7f32041006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 shutdown_connections 2024-09-13T13:25:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:25:55.300+0000 7f320a6ac640 1 -- 172.21.15.43:0/2323925058 wait complete. 2024-09-13T13:25:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T13:25:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T13:25:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:55.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:55.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:55.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:25:55.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T13:25:55.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:55.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:25:55.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:25:55.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T13:25:55.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:55.957 DEBUG:teuthology.orchestra.run.smithi043:osd.1> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.1.service 2024-09-13T13:25:55.960 INFO:tasks.cephadm:Deploying osd.2 on smithi043 with /dev/vg_nvme/lv_2... 2024-09-13T13:25:55.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-13T13:25:56.692 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:25:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:56 smithi043 ceph-mon[30106]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:56.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:56 smithi043 ceph-mon[40255]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:56.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:56.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:56 smithi189 ceph-mon[32912]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:57.605 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:25:57.605 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:25:57.605 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[40255]: osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] boot 2024-09-13T13:25:57.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[40255]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T13:25:57.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:57.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:25:57.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:25:57.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[30106]: osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] boot 2024-09-13T13:25:57.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[30106]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T13:25:57.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:57 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:25:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:57 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:25:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:57 smithi189 ceph-mon[32912]: osd.1 [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] boot 2024-09-13T13:25:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:57 smithi189 ceph-mon[32912]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T13:25:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:25:58.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:58 smithi043 ceph-mon[30106]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:58.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:25:58 smithi043 ceph-mon[30106]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T13:25:58.793 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:58 smithi043 ceph-mon[40255]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:58.793 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:25:58 smithi043 ceph-mon[40255]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T13:25:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:58 smithi189 ceph-mon[32912]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-13T13:25:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:25:58 smithi189 ceph-mon[32912]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T13:26:00.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:00.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi043 2024-09-13T13:26:00.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-13T13:26:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:00 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi043 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-13T13:26:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi043 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:00.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:00.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:00 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:01.050 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:26:01.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-09-13T13:26:01.341 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:26:01.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 -- 172.21.15.43:0/216245828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958076f00 msgr2=0x7f8958075560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:01.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 --2- 172.21.15.43:0/216245828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958076f00 0x7f8958075560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8948009900 tx=0x7f894802a410 comp rx=0 tx=0).stop 2024-09-13T13:26:01.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 -- 172.21.15.43:0/216245828 shutdown_connections 2024-09-13T13:26:01.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 --2- 172.21.15.43:0/216245828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f895810a9c0 0x7f895810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:01.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 --2- 172.21.15.43:0/216245828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8958075aa0 0x7f8958075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:01.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 --2- 172.21.15.43:0/216245828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958076f00 0x7f8958075560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 -- 172.21.15.43:0/216245828 >> 172.21.15.43:0/216245828 conn(0x7f89580fe2c0 msgr2=0x7f8958100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:26:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 -- 172.21.15.43:0/216245828 shutdown_connections 2024-09-13T13:26:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 -- 172.21.15.43:0/216245828 wait complete. 2024-09-13T13:26:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.852+0000 7f895ce1a640 1 Processor -- start 2024-09-13T13:26:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895ce1a640 1 -- start start 2024-09-13T13:26:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895ce1a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958075aa0 0x7f895819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:01.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895ce1a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8958076f00 0x7f895819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895ce1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895810a9c0 0x7f895819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895ce1a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f895810fd00 con 0x7f895810a9c0 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895ce1a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f895810fb80 con 0x7f8958076f00 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895ce1a640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f895810fe80 con 0x7f8958075aa0 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958075aa0 0x7f895819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958075aa0 0x7f895819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:45578/0 (socket says 172.21.15.43:45578) 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 -- 172.21.15.43:0/3685946409 learned_addr learned my addr 172.21.15.43:0/3685946409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:26:01.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f8956d9d640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895810a9c0 0x7f895819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f8955d9b640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8958076f00 0x7f895819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 -- 172.21.15.43:0/3685946409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8958076f00 msgr2=0x7f895819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8958076f00 0x7f895819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 -- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895810a9c0 msgr2=0x7f895819b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895810a9c0 0x7f895819b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 -- 172.21.15.43:0/3685946409 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895819c080 con 0x7f8958075aa0 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f8955d9b640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8958076f00 0x7f895819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f8956d9d640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895810a9c0 0x7f895819b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.853+0000 7f895659c640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958075aa0 0x7f895819a550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8948005940 tx=0x7f89480042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.854+0000 7f89477fe640 1 -- 172.21.15.43:0/3685946409 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894803f070 con 0x7f8958075aa0 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.854+0000 7f89477fe640 1 -- 172.21.15.43:0/3685946409 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f89480053d0 con 0x7f8958075aa0 2024-09-13T13:26:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.854+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895819c310 con 0x7f8958075aa0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.854+0000 7f89477fe640 1 -- 172.21.15.43:0/3685946409 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948035030 con 0x7f8958075aa0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.854+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89581a2a50 con 0x7f8958075aa0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.855+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8958075f20 con 0x7f8958075aa0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.855+0000 7f89477fe640 1 -- 172.21.15.43:0/3685946409 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 15) ==== 103023+0+0 (secure 0 0 0) 0x7f8948042070 con 0x7f8958075aa0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.856+0000 7f89477fe640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f894007ac90 0x7f894007d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.856+0000 7f89477fe640 1 -- 172.21.15.43:0/3685946409 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 2199+0+0 (secure 0 0 0) 0x7f89480bd920 con 0x7f8958075aa0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.856+0000 7f8955d9b640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f894007ac90 0x7f894007d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.856+0000 7f8955d9b640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f894007ac90 0x7f894007d150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f894c009c30 tx=0x7f894c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:26:01.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:01.861+0000 7f89477fe640 1 -- 172.21.15.43:0/3685946409 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f894808a190 con 0x7f8958075aa0 2024-09-13T13:26:02.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:02.058+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f895802e730 con 0x7f894007ac90 2024-09-13T13:26:02.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[30106]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:02.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:02.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:02.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:02.612 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[40255]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:02.612 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:02.612 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:02.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:02.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:02 smithi189 ceph-mon[32912]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:02.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:02.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:02.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:03.480 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:03 smithi043 ceph-mon[30106]: from='client.24164 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:26:03.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:03 smithi043 ceph-mon[40255]: from='client.24164 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:26:03.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:03 smithi189 ceph-mon[32912]: from='client.24164 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:26:04.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:04 smithi189 ceph-mon[32912]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:04.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:04 smithi043 ceph-mon[30106]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:04.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:04 smithi043 ceph-mon[40255]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/107750637' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"} : dispatch 2024-09-13T13:26:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"} : dispatch 2024-09-13T13:26:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"}]': finished 2024-09-13T13:26:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:05 smithi189 ceph-mon[32912]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T13:26:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:05 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2334694612' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:26:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/107750637' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"} : dispatch 2024-09-13T13:26:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"} : dispatch 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"}]': finished 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[30106]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2334694612' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/107750637' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"} : dispatch 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"} : dispatch 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cad8dbd-500f-46d9-9f70-27d4b12c8835"}]': finished 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[40255]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2334694612' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:26:06.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:06 smithi043 ceph-mon[30106]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:06 smithi043 ceph-mon[40255]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:06 smithi189 ceph-mon[32912]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:08.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:08 smithi043 ceph-mon[30106]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:08.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:08 smithi043 ceph-mon[40255]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:08 smithi189 ceph-mon[32912]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:10.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:10 smithi043 ceph-mon[30106]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:10.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:10 smithi043 ceph-mon[40255]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:10 smithi189 ceph-mon[32912]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:12.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[30106]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:12.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-13T13:26:12.513 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:12.513 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[30106]: Deploying daemon osd.2 on smithi043 2024-09-13T13:26:12.513 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[40255]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:12.513 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-13T13:26:12.513 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:12.513 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:12 smithi043 ceph-mon[40255]: Deploying daemon osd.2 on smithi043 2024-09-13T13:26:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:12 smithi189 ceph-mon[32912]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:12 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-13T13:26:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:12 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:12 smithi189 ceph-mon[32912]: Deploying daemon osd.2 on smithi043 2024-09-13T13:26:14.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:14 smithi043 ceph-mon[30106]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:14.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:14 smithi043 ceph-mon[40255]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:14 smithi189 ceph-mon[32912]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:16 smithi043 ceph-mon[30106]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:16.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:16 smithi043 ceph-mon[40255]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:16 smithi189 ceph-mon[32912]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:18 smithi043 ceph-mon[30106]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:18 smithi043 ceph-mon[40255]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:18 smithi189 ceph-mon[32912]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:20 smithi043 ceph-mon[30106]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:20 smithi043 ceph-mon[40255]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:20.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:20 smithi189 ceph-mon[32912]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[40255]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[30106]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:22.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:22.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:22.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:22 smithi189 ceph-mon[32912]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:24.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:24 smithi043 ceph-mon[30106]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:24.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:24 smithi043 ceph-mon[30106]: from='osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T13:26:24.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:24 smithi043 ceph-mon[30106]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T13:26:24.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:24 smithi043 ceph-mon[40255]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:24.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:24 smithi043 ceph-mon[40255]: from='osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T13:26:24.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:24 smithi043 ceph-mon[40255]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T13:26:24.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:24 smithi189 ceph-mon[32912]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:24.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:24 smithi189 ceph-mon[32912]: from='osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T13:26:24.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:24 smithi189 ceph-mon[32912]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T13:26:24.965 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 2 on host 'smithi043' 2024-09-13T13:26:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.963+0000 7f89477fe640 1 -- 172.21.15.43:0/3685946409 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f895802e730 con 0x7f894007ac90 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f894007ac90 msgr2=0x7f894007d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f894007ac90 0x7f894007d150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f894c009c30 tx=0x7f894c005f90 comp rx=0 tx=0).stop 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958075aa0 msgr2=0x7f895819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958075aa0 0x7f895819a550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8948005940 tx=0x7f89480042f0 comp rx=0 tx=0).stop 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 shutdown_connections 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f894007ac90 0x7f894007d150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895810a9c0 0x7f895819b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8958076f00 0x7f895819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 --2- 172.21.15.43:0/3685946409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8958075aa0 0x7f895819a550 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.966+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 >> 172.21.15.43:0/3685946409 conn(0x7f89580fe2c0 msgr2=0x7f89581006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:26:24.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.967+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 shutdown_connections 2024-09-13T13:26:24.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:24.967+0000 7f895ce1a640 1 -- 172.21.15.43:0/3685946409 wait complete. 2024-09-13T13:26:25.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T13:26:25.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T13:26:25.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:25.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T13:26:25.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T13:26:25.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:25.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:25.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:25.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:25.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:25.596 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:25.596 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:26:25.596 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T13:26:25.596 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:25 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:25.625 DEBUG:teuthology.orchestra.run.smithi043:osd.2> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.2.service 2024-09-13T13:26:25.629 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_1... 2024-09-13T13:26:25.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-13T13:26:25.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T13:26:25.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T13:26:25.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T13:26:25.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:25 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:26:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:26 smithi043 ceph-mon[30106]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:26 smithi043 ceph-mon[40255]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:26.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:26 smithi189 ceph-mon[32912]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:26.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:26.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[30106]: osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] boot 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[30106]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:26:27.798 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[40255]: osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] boot 2024-09-13T13:26:27.799 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[40255]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T13:26:27.799 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:27 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:26:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:27 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:26:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:27 smithi189 ceph-mon[32912]: osd.2 [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] boot 2024-09-13T13:26:27.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:27 smithi189 ceph-mon[32912]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T13:26:27.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:26:28.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:28 smithi189 ceph-mon[32912]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:28.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:28 smithi189 ceph-mon[32912]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T13:26:28.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:28 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:26:28.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:28 smithi043 ceph-mon[30106]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:28.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:28 smithi043 ceph-mon[30106]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T13:26:28.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:26:28.938 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:28 smithi043 ceph-mon[40255]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T13:26:28.938 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:28 smithi043 ceph-mon[40255]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T13:26:28.938 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:28 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:26:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T13:26:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T13:26:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:26:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:29.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T13:26:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T13:26:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:26:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T13:26:29.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:29.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:30.586 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T13:26:30.586 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.586 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60209]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.587 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60209]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.587 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60212]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T13:26:30.587 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60212]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.587 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60212]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.587 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60212]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi043 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:30.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T13:26:30.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60227]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60227]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60227]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60227]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60230]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60230]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60230]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60230]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi043 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:30.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:30.592 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:30.592 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:30.592 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:30.592 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T13:26:30.592 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.592 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60215]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.592 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60215]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.592 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60218]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T13:26:30.592 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60218]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60218]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60218]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60221]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60221]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60221]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60221]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60224]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60224]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60224]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.593 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60224]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.708 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:26:30.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-09-13T13:26:30.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35036]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T13:26:30.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35036]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35036]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35036]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35039]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35039]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35039]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 sudo[35039]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi043 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-13T13:26:30.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:30.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:30.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:30.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:30.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:26:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60235]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60235]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T13:26:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T13:26:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60238]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T13:26:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:30 smithi043 sudo[60238]: pam_unix(sudo:session): session closed for user root 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 -- 172.21.15.43:0/2917417202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0103440 msgr2=0x7f9eb0109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/2917417202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0103440 0x7f9eb0109dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009cd0 tx=0x7f9ea802a8f0 comp rx=0 tx=0).stop 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 -- 172.21.15.43:0/2917417202 shutdown_connections 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/2917417202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0103440 0x7f9eb0109dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/2917417202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9eb0102a80 0x7f9eb0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/2917417202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9eb0101920 0x7f9eb0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 -- 172.21.15.43:0/2917417202 >> 172.21.15.43:0/2917417202 conn(0x7f9eb00fd1d0 msgr2=0x7f9eb00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 -- 172.21.15.43:0/2917417202 shutdown_connections 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.444+0000 7f9eb7a81640 1 -- 172.21.15.43:0/2917417202 wait complete. 2024-09-13T13:26:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 Processor -- start 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 -- start start 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0101920 0x7f9eb0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9eb0102a80 0x7f9eb01978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9eb0103440 0x7f9eb01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb010ca00 con 0x7f9eb0101920 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb010c880 con 0x7f9eb0103440 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb7a81640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb010cb80 con 0x7f9eb0102a80 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0101920 0x7f9eb0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0101920 0x7f9eb0197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39106/0 (socket says 172.21.15.43:39106) 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb501c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9eb0102a80 0x7f9eb01978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 -- 172.21.15.43:0/3727215001 learned_addr learned my addr 172.21.15.43:0/3727215001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb601e640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9eb0103440 0x7f9eb01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 -- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9eb0102a80 msgr2=0x7f9eb01978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9eb0102a80 0x7f9eb01978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 -- 172.21.15.43:0/3727215001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9eb0103440 msgr2=0x7f9eb01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9eb0103440 0x7f9eb01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb581d640 1 -- 172.21.15.43:0/3727215001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb0198d80 con 0x7f9eb0101920 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb601e640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9eb0103440 0x7f9eb01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.445+0000 7f9eb501c640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9eb0102a80 0x7f9eb01978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.446+0000 7f9eb581d640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0101920 0x7f9eb0197250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00ea60 tx=0x7f9eac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.446+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00cde0 con 0x7f9eb0101920 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.446+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb0199070 con 0x7f9eb0101920 2024-09-13T13:26:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.446+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9eac004470 con 0x7f9eb0101920 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.446+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac010430 con 0x7f9eb0101920 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.446+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb019f750 con 0x7f9eb0101920 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.447+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb0101d20 con 0x7f9eb0101920 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.448+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) ==== 103023+0+0 (secure 0 0 0) 0x7f9eac003f60 con 0x7f9eb0101920 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.448+0000 7f9ea6ffd640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9e9c07ac90 0x7f9e9c07d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.448+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3017+0+0 (secure 0 0 0) 0x7f9eac09d3e0 con 0x7f9eb0101920 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.448+0000 7f9eb501c640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9e9c07ac90 0x7f9e9c07d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.449+0000 7f9eb501c640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9e9c07ac90 0x7f9e9c07d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000bfd0 tx=0x7f9ea000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:26:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.453+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f9eac0654c0 con 0x7f9eb0101920 2024-09-13T13:26:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.648+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9eb0063590 con 0x7f9e9c07ac90 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:31.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[40255]: osdmap e22: 3 total, 3 up, 3 in 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:31 smithi043 ceph-mon[30106]: osdmap e22: 3 total, 3 up, 3 in 2024-09-13T13:26:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T13:26:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:26:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:26:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:26:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T13:26:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:31 smithi189 ceph-mon[32912]: osdmap e22: 3 total, 3 up, 3 in 2024-09-13T13:26:31.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:31.998+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f9eac061030 con 0x7f9eb0101920 2024-09-13T13:26:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:26:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:32.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T13:26:32.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: Cluster is now healthy 2024-09-13T13:26:32.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:32 smithi189 ceph-mon[32912]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-13T13:26:32.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:32.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:26:32.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:32.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:32.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: Cluster is now healthy 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[30106]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: Cluster is now healthy 2024-09-13T13:26:32.973 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:32 smithi043 ceph-mon[40255]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-13T13:26:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:34 smithi189 ceph-mon[32912]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3358297245' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"} : dispatch 2024-09-13T13:26:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"} : dispatch 2024-09-13T13:26:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"}]': finished 2024-09-13T13:26:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:34 smithi189 ceph-mon[32912]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T13:26:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:34 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[30106]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3358297245' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"} : dispatch 2024-09-13T13:26:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"} : dispatch 2024-09-13T13:26:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"}]': finished 2024-09-13T13:26:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[30106]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T13:26:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:34.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[40255]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:34.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3358297245' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"} : dispatch 2024-09-13T13:26:34.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"} : dispatch 2024-09-13T13:26:34.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c24c101f-2a35-4ed4-8cf3-6ec3824a78de"}]': finished 2024-09-13T13:26:34.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[40255]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T13:26:34.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:34 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1163729543' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:26:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1163729543' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:26:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1163729543' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:26:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:36 smithi189 ceph-mon[32912]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:36 smithi043 ceph-mon[40255]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:36.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:36 smithi043 ceph-mon[30106]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:38.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:38 smithi043 ceph-mon[30106]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:38.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:38 smithi043 ceph-mon[40255]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:38 smithi189 ceph-mon[32912]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:40.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:40 smithi043 ceph-mon[30106]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:40.834 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:40 smithi043 ceph-mon[40255]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:40 smithi189 ceph-mon[32912]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:41 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-13T13:26:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:41 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:41 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-13T13:26:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:41 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:41 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-13T13:26:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:41 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:42.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:42 smithi043 ceph-mon[30106]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:42.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:42 smithi043 ceph-mon[30106]: Deploying daemon osd.3 on smithi043 2024-09-13T13:26:42.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:42 smithi043 ceph-mon[40255]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:42.850 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:42 smithi043 ceph-mon[40255]: Deploying daemon osd.3 on smithi043 2024-09-13T13:26:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:42 smithi189 ceph-mon[32912]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:42 smithi189 ceph-mon[32912]: Deploying daemon osd.3 on smithi043 2024-09-13T13:26:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:44 smithi189 ceph-mon[32912]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:44.907 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:44 smithi043 ceph-mon[30106]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:44.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:44 smithi043 ceph-mon[40255]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:46 smithi189 ceph-mon[32912]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:46.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:46 smithi043 ceph-mon[30106]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:46.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:46 smithi043 ceph-mon[40255]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:48 smithi043 ceph-mon[30106]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:48 smithi043 ceph-mon[40255]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:48 smithi189 ceph-mon[32912]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:49 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:49 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:49 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:49.945 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:49 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:49.945 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:49 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:49.945 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:49 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:49.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:49 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:49.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:49 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:49.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:49 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:50.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:50 smithi189 ceph-mon[32912]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:50 smithi043 ceph-mon[30106]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:50 smithi043 ceph-mon[40255]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:52 smithi189 ceph-mon[32912]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[40255]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[30106]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:54 smithi043 ceph-mon[40255]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:54 smithi043 ceph-mon[40255]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-13T13:26:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:54 smithi043 ceph-mon[30106]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:54 smithi043 ceph-mon[30106]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-13T13:26:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-09-13T13:26:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.848+0000 7f9ea6ffd640 1 -- 172.21.15.43:0/3727215001 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9eb0063590 con 0x7f9e9c07ac90 2024-09-13T13:26:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9e9c07ac90 msgr2=0x7f9e9c07d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9e9c07ac90 0x7f9e9c07d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000bfd0 tx=0x7f9ea000b390 comp rx=0 tx=0).stop 2024-09-13T13:26:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0101920 msgr2=0x7f9eb0197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0101920 0x7f9eb0197250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00ea60 tx=0x7f9eac00ef20 comp rx=0 tx=0).stop 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 shutdown_connections 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9e9c07ac90 0x7f9e9c07d150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9eb0103440 0x7f9eb01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9eb0102a80 0x7f9eb01978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.852+0000 7f9eb7a81640 1 --2- 172.21.15.43:0/3727215001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb0101920 0x7f9eb0197250 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.853+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 >> 172.21.15.43:0/3727215001 conn(0x7f9eb00fd1d0 msgr2=0x7f9eb00fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.853+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 shutdown_connections 2024-09-13T13:26:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:26:54.853+0000 7f9eb7a81640 1 -- 172.21.15.43:0/3727215001 wait complete. 2024-09-13T13:26:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:54 smithi189 ceph-mon[32912]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:54 smithi189 ceph-mon[32912]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-13T13:26:55.464 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.3.service 2024-09-13T13:26:55.467 INFO:tasks.cephadm:Deploying osd.4 on smithi189 with /dev/vg_nvme/lv_4... 2024-09-13T13:26:55.467 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-13T13:26:55.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[30106]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[30106]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[30106]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[40255]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[40255]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T13:26:55.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[40255]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:55.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:55.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:55.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:55.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:55.689 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:26:55.603+0000 7f1dac8a8640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T13:26:55.689 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:26:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:26:55.606+0000 7f1da86cf640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T13:26:55.709 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:26:55.830 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:55 smithi189 ceph-mon[32912]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T13:26:55.830 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:55 smithi189 ceph-mon[32912]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T13:26:55.830 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:55 smithi189 ceph-mon[32912]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-13T13:26:55.830 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:55.830 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:26:55.830 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:55.830 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:56.785 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[30106]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:56.785 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[30106]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:26:56.785 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[30106]: osdmap e25: 4 total, 3 up, 4 in 2024-09-13T13:26:56.785 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:56.785 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:56.785 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[40255]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:56.785 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[40255]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:26:56.786 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[40255]: osdmap e25: 4 total, 3 up, 4 in 2024-09-13T13:26:56.786 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:56.786 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:56.887 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:56 smithi189 ceph-mon[32912]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:56.887 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:56 smithi189 ceph-mon[32912]: from='osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-13T13:26:56.887 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:56 smithi189 ceph-mon[32912]: osdmap e25: 4 total, 3 up, 4 in 2024-09-13T13:26:56.888 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:56.888 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:57 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:26:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:57 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:26:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:57 smithi189 ceph-mon[32912]: osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] boot 2024-09-13T13:26:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:57 smithi189 ceph-mon[32912]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T13:26:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:26:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:26:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[30106]: osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] boot 2024-09-13T13:26:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[30106]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T13:26:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:57.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:26:57.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:26:57.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[40255]: osd.3 [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] boot 2024-09-13T13:26:57.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[40255]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T13:26:57.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:26:58.573 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:26:58.597 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:58.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:58.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:58 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.841 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:26:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-13T13:26:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T13:26:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:26:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:26:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:58 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:26:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 -- 172.21.15.189:0/3227247569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18102a10 msgr2=0x7f2f18102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/3227247569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18102a10 0x7f2f18102e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009930 tx=0x7f2f0802a420 comp rx=0 tx=0).stop 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 -- 172.21.15.189:0/3227247569 shutdown_connections 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/3227247569 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f18104530 0x7f2f1810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/3227247569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f18103b70 0x7f2f18103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/3227247569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18102a10 0x7f2f18102e10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 -- 172.21.15.189:0/3227247569 >> 172.21.15.189:0/3227247569 conn(0x7f2f180fe2c0 msgr2=0x7f2f18100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 -- 172.21.15.189:0/3227247569 shutdown_connections 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.315+0000 7f2f1db5c640 1 -- 172.21.15.189:0/3227247569 wait complete. 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 Processor -- start 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 -- start start 2024-09-13T13:26:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f18102a10 0x7f2f18076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18103b70 0x7f2f18075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f18104530 0x7f2f18075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1810dbb0 con 0x7f2f18104530 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1810da30 con 0x7f2f18103b70 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f1db5c640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1810dd30 con 0x7f2f18102a10 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18103b70 0x7f2f18075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18103b70 0x7f2f18075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:33226/0 (socket says 172.21.15.189:33226) 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.316+0000 7f2f16ffd640 1 -- 172.21.15.189:0/284627813 learned_addr learned my addr 172.21.15.189:0/284627813 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f17fff640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f18104530 0x7f2f18075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:59.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f177fe640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f18102a10 0x7f2f18076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f16ffd640 1 -- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f18102a10 msgr2=0x7f2f18076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f16ffd640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f18102a10 0x7f2f18076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f16ffd640 1 -- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f18104530 msgr2=0x7f2f18075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f16ffd640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f18104530 0x7f2f18075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f16ffd640 1 -- 172.21.15.189:0/284627813 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f18076340 con 0x7f2f18103b70 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f17fff640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f18104530 0x7f2f18075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f177fe640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f18102a10 0x7f2f18076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f16ffd640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18103b70 0x7f2f18075540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c890 tx=0x7f2f0c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f14ff9640 1 -- 172.21.15.189:0/284627813 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c004220 con 0x7f2f18103b70 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f14ff9640 1 -- 172.21.15.189:0/284627813 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f2f0c007e00 con 0x7f2f18103b70 2024-09-13T13:26:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f181a2460 con 0x7f2f18103b70 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f14ff9640 1 -- 172.21.15.189:0/284627813 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0047a0 con 0x7f2f18103b70 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.317+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181a2a40 con 0x7f2f18103b70 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.318+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f18102e10 con 0x7f2f18103b70 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.319+0000 7f2f14ff9640 1 -- 172.21.15.189:0/284627813 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f2f0c0079b0 con 0x7f2f18103b70 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.319+0000 7f2f14ff9640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2f0007ad20 0x7f2f0007d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.319+0000 7f2f14ff9640 1 -- 172.21.15.189:0/284627813 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3449+0+0 (secure 0 0 0) 0x7f2f0c09d420 con 0x7f2f18103b70 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.319+0000 7f2f177fe640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2f0007ad20 0x7f2f0007d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.320+0000 7f2f177fe640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2f0007ad20 0x7f2f0007d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005ab0 tx=0x7f2f08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:26:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.324+0000 7f2f14ff9640 1 -- 172.21.15.189:0/284627813 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f2f0c014030 con 0x7f2f18103b70 2024-09-13T13:26:59.509 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:26:59.507+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2f1802e840 con 0x7f2f0007ad20 2024-09-13T13:26:59.881 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:59 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi043 2024-09-13T13:26:59.882 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:59 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-13T13:26:59.882 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:59.882 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:59.882 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:26:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi043 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi043 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:26:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:26:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:26:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:00 smithi189 ceph-mon[32912]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:00 smithi189 ceph-mon[32912]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:27:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:00 smithi043 ceph-mon[30106]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:00 smithi043 ceph-mon[30106]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:27:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:00 smithi043 ceph-mon[40255]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:00 smithi043 ceph-mon[40255]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:27:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:02 smithi189 ceph-mon[32912]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/2445538118' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"} : dispatch 2024-09-13T13:27:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"} : dispatch 2024-09-13T13:27:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"}]': finished 2024-09-13T13:27:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:02 smithi189 ceph-mon[32912]: osdmap e28: 5 total, 4 up, 5 in 2024-09-13T13:27:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/2154841955' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[30106]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/2445538118' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"} : dispatch 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"} : dispatch 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"}]': finished 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[30106]: osdmap e28: 5 total, 4 up, 5 in 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/2154841955' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:27:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[40255]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/2445538118' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"} : dispatch 2024-09-13T13:27:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"} : dispatch 2024-09-13T13:27:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d500438-0ae5-4c98-bd0b-ed0955ef9aea"}]': finished 2024-09-13T13:27:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[40255]: osdmap e28: 5 total, 4 up, 5 in 2024-09-13T13:27:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/2154841955' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:27:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:04 smithi189 ceph-mon[32912]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:04 smithi043 ceph-mon[30106]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:04.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:04 smithi043 ceph-mon[40255]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:06 smithi189 ceph-mon[32912]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:06 smithi043 ceph-mon[40255]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:06 smithi043 ceph-mon[30106]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:08.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:08 smithi189 ceph-mon[32912]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:08 smithi043 ceph-mon[40255]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:08 smithi043 ceph-mon[30106]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:09.768 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:09 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-13T13:27:09.768 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:09 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:09 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-13T13:27:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:09 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:09 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-13T13:27:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:09 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:10.761 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:10 smithi189 ceph-mon[32912]: Deploying daemon osd.4 on smithi189 2024-09-13T13:27:10.761 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:10 smithi189 ceph-mon[32912]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:10 smithi043 ceph-mon[40255]: Deploying daemon osd.4 on smithi189 2024-09-13T13:27:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:10 smithi043 ceph-mon[40255]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:10 smithi043 ceph-mon[30106]: Deploying daemon osd.4 on smithi189 2024-09-13T13:27:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:10 smithi043 ceph-mon[30106]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:12.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:12 smithi189 ceph-mon[32912]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:12 smithi043 ceph-mon[30106]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:12 smithi043 ceph-mon[40255]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:14 smithi189 ceph-mon[32912]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:14 smithi043 ceph-mon[30106]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:14 smithi043 ceph-mon[40255]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:16 smithi189 ceph-mon[32912]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:16 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:16 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:16 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[30106]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[40255]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:16 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:18 smithi189 ceph-mon[32912]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:18 smithi043 ceph-mon[30106]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:18 smithi043 ceph-mon[40255]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[40255]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[30106]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:20.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:20.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:20 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:20 smithi189 ceph-mon[32912]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:20 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:20 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:20 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:20 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:20 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:22.193 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 4 on host 'smithi189' 2024-09-13T13:27:22.194 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.192+0000 7f2f14ff9640 1 -- 172.21.15.189:0/284627813 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f1802e840 con 0x7f2f0007ad20 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2f0007ad20 msgr2=0x7f2f0007d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2f0007ad20 0x7f2f0007d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005ab0 tx=0x7f2f08005fd0 comp rx=0 tx=0).stop 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18103b70 msgr2=0x7f2f18075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18103b70 0x7f2f18075540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c890 tx=0x7f2f0c00cd50 comp rx=0 tx=0).stop 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 shutdown_connections 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2f0007ad20 0x7f2f0007d1e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f18104530 0x7f2f18075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:22.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f18103b70 0x7f2f18075540 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:22.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 --2- 172.21.15.189:0/284627813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f18102a10 0x7f2f18076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:22.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 >> 172.21.15.189:0/284627813 conn(0x7f2f180fe2c0 msgr2=0x7f2f180fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:27:22.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 shutdown_connections 2024-09-13T13:27:22.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:22.195+0000 7f2f1db5c640 1 -- 172.21.15.189:0/284627813 wait complete. 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[30106]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[30106]: from='osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[30106]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[40255]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[40255]: from='osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T13:27:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[40255]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T13:27:22.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:22.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:22.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:22 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:22.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:22 smithi189 ceph-mon[32912]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:22.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:22 smithi189 ceph-mon[32912]: from='osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T13:27:22.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:22 smithi189 ceph-mon[32912]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T13:27:22.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:22.499 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:22.499 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:22 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:22.819 DEBUG:teuthology.orchestra.run.smithi189:osd.4> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.4.service 2024-09-13T13:27:22.821 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-09-13T13:27:22.821 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-13T13:27:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[30106]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T13:27:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[30106]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T13:27:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[30106]: from='osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[30106]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[40255]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T13:27:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[40255]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T13:27:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[40255]: from='osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:23 smithi043 ceph-mon[40255]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:23.592 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:27:23.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:23 smithi189 ceph-mon[32912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T13:27:23.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:23 smithi189 ceph-mon[32912]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T13:27:23.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:23 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:23.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:23 smithi189 ceph-mon[32912]: from='osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:23.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:23 smithi189 ceph-mon[32912]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:23.649 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:27:23 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:27:23.239+0000 7f6dc3ea5640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T13:27:23.649 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:27:23 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:27:23.244+0000 7f6dbfccc640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T13:27:24.511 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:24 smithi189 ceph-mon[32912]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:24.511 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:24 smithi189 ceph-mon[32912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:27:24.511 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:24 smithi189 ceph-mon[32912]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T13:27:24.511 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:24.512 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[30106]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[30106]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[30106]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[40255]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[40255]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[40255]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T13:27:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:25.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:25 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:27:25.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:25 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:27:25.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:25 smithi189 ceph-mon[32912]: osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] boot 2024-09-13T13:27:25.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:25 smithi189 ceph-mon[32912]: osdmap e31: 5 total, 5 up, 5 in 2024-09-13T13:27:25.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:25 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:25.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:25 smithi189 ceph-mon[32912]: osdmap e32: 5 total, 5 up, 5 in 2024-09-13T13:27:25.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:27:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:27:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[30106]: osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] boot 2024-09-13T13:27:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[30106]: osdmap e31: 5 total, 5 up, 5 in 2024-09-13T13:27:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[30106]: osdmap e32: 5 total, 5 up, 5 in 2024-09-13T13:27:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:27:25.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:27:25.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[40255]: osd.4 [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] boot 2024-09-13T13:27:25.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[40255]: osdmap e31: 5 total, 5 up, 5 in 2024-09-13T13:27:25.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:27:25.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:25 smithi043 ceph-mon[40255]: osdmap e32: 5 total, 5 up, 5 in 2024-09-13T13:27:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:26 smithi189 ceph-mon[32912]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T13:27:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[30106]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:26.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:26.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:26.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:26.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:26 smithi043 ceph-mon[40255]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T13:27:27.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:27 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi189 2024-09-13T13:27:27.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:27 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-13T13:27:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:27 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi189 2024-09-13T13:27:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:27 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-13T13:27:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:27 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi189 2024-09-13T13:27:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:27 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-13T13:27:28.012 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:27:28.043 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-09-13T13:27:28.292 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:27:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:28 smithi189 ceph-mon[32912]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:28 smithi043 ceph-mon[30106]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:28 smithi043 ceph-mon[40255]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:28.803 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 -- 172.21.15.189:0/3869469723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8103b60 msgr2=0x7f7ed8103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3869469723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8103b60 0x7f7ed8103fe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009930 tx=0x7f7ec402a420 comp rx=0 tx=0).stop 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 -- 172.21.15.189:0/3869469723 shutdown_connections 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3869469723 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ed8104520 0x7f7ed810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3869469723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8103b60 0x7f7ed8103fe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3869469723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed8102a00 0x7f7ed8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 -- 172.21.15.189:0/3869469723 >> 172.21.15.189:0/3869469723 conn(0x7f7ed80fe2f0 msgr2=0x7f7ed8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 -- 172.21.15.189:0/3869469723 shutdown_connections 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.795+0000 7f7edefa2640 1 -- 172.21.15.189:0/3869469723 wait complete. 2024-09-13T13:27:28.804 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 Processor -- start 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 -- start start 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ed8102a00 0x7f7ed8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed8103b60 0x7f7ed81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8104520 0x7f7ed8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed810dbc0 con 0x7f7ed8103b60 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed810da40 con 0x7f7ed8104520 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edefa2640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed810dd40 con 0x7f7ed8102a00 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8104520 0x7f7ed8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8104520 0x7f7ed8199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:36338/0 (socket says 172.21.15.189:36338) 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 -- 172.21.15.189:0/3182793500 learned_addr learned my addr 172.21.15.189:0/3182793500 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:27:28.805 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edcd3e640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ed8102a00 0x7f7ed8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 -- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ed8102a00 msgr2=0x7f7ed8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ed8102a00 0x7f7ed8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 -- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed8103b60 msgr2=0x7f7ed81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7ed7fff640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed8103b60 0x7f7ed81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed8103b60 0x7f7ed81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7edd53f640 1 -- 172.21.15.189:0/3182793500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8199ec0 con 0x7f7ed8104520 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.796+0000 7f7ed7fff640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed8103b60 0x7f7ed81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.797+0000 7f7edd53f640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8104520 0x7f7ed8199620 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b970 tx=0x7f7ed000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.797+0000 7f7ed5ffb640 1 -- 172.21.15.189:0/3182793500 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0019070 con 0x7f7ed8104520 2024-09-13T13:27:28.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.797+0000 7f7ed5ffb640 1 -- 172.21.15.189:0/3182793500 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7ed0014440 con 0x7f7ed8104520 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.797+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed819a1b0 con 0x7f7ed8104520 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.797+0000 7f7ed5ffb640 1 -- 172.21.15.189:0/3182793500 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0014e60 con 0x7f7ed8104520 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.797+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a0890 con 0x7f7ed8104520 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.798+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed8102e00 con 0x7f7ed8104520 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.799+0000 7f7ed5ffb640 1 -- 172.21.15.189:0/3182793500 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f7ed001c070 con 0x7f7ed8104520 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.799+0000 7f7ed5ffb640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7ec807ad20 0x7f7ec807d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.799+0000 7f7ed5ffb640 1 -- 172.21.15.189:0/3182793500 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 3991+0+0 (secure 0 0 0) 0x7f7ed009ca40 con 0x7f7ed8104520 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.799+0000 7f7edcd3e640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7ec807ad20 0x7f7ec807d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.800+0000 7f7edcd3e640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7ec807ad20 0x7f7ec807d1e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0058c0 tx=0x7f7ecc005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:27:28.807 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.803+0000 7f7ed5ffb640 1 -- 172.21.15.189:0/3182793500 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f7ed0068ad0 con 0x7f7ed8104520 2024-09-13T13:27:28.987 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:28.986+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7ed802e8a0 con 0x7f7ec807ad20 2024-09-13T13:27:29.606 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:27:29.606 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:27:29.606 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:27:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:27:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:27:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:27:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:30.351 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:30 smithi189 ceph-mon[32912]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:27:30.351 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:30 smithi189 ceph-mon[32912]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-13T13:27:30.662 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:30 smithi043 ceph-mon[30106]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:27:30.662 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:30 smithi043 ceph-mon[30106]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-13T13:27:30.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:30 smithi043 ceph-mon[40255]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:27:30.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:30 smithi043 ceph-mon[40255]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-13T13:27:32.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:32 smithi189 ceph-mon[32912]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T13:27:32.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/442064979' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"} : dispatch 2024-09-13T13:27:32.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"} : dispatch 2024-09-13T13:27:32.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"}]': finished 2024-09-13T13:27:32.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:32 smithi189 ceph-mon[32912]: osdmap e34: 6 total, 5 up, 6 in 2024-09-13T13:27:32.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:32 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:32.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/2324603016' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:27:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[30106]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/442064979' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"} : dispatch 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"} : dispatch 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"}]': finished 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[30106]: osdmap e34: 6 total, 5 up, 6 in 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/2324603016' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[40255]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T13:27:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/442064979' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"} : dispatch 2024-09-13T13:27:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"} : dispatch 2024-09-13T13:27:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "141633af-2b7a-4224-8c9e-43d82589627b"}]': finished 2024-09-13T13:27:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[40255]: osdmap e34: 6 total, 5 up, 6 in 2024-09-13T13:27:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/2324603016' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:27:34.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:34 smithi043 ceph-mon[30106]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-13T13:27:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:34 smithi043 ceph-mon[40255]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-13T13:27:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:34 smithi189 ceph-mon[32912]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-13T13:27:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:36 smithi189 ceph-mon[32912]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-13T13:27:36.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:36 smithi043 ceph-mon[30106]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-13T13:27:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:36 smithi043 ceph-mon[40255]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-13T13:27:38.646 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:38 smithi189 ceph-mon[32912]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:27:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:38 smithi043 ceph-mon[30106]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:27:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:38 smithi043 ceph-mon[40255]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:27:39.548 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:39 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-13T13:27:39.548 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:39 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:39 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-13T13:27:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:39 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:39 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-13T13:27:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:39 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:40.377 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:40 smithi189 ceph-mon[32912]: Deploying daemon osd.5 on smithi189 2024-09-13T13:27:40.377 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:40 smithi189 ceph-mon[32912]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:40 smithi043 ceph-mon[30106]: Deploying daemon osd.5 on smithi189 2024-09-13T13:27:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:40 smithi043 ceph-mon[30106]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:40 smithi043 ceph-mon[40255]: Deploying daemon osd.5 on smithi189 2024-09-13T13:27:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:40 smithi043 ceph-mon[40255]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:42.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:42 smithi189 ceph-mon[32912]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:42 smithi043 ceph-mon[30106]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:42.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:42 smithi043 ceph-mon[40255]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:44 smithi043 ceph-mon[30106]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:44 smithi043 ceph-mon[40255]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:44.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:44 smithi189 ceph-mon[32912]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:46 smithi189 ceph-mon[32912]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:46 smithi043 ceph-mon[30106]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:46.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:46 smithi043 ceph-mon[40255]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:47 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:47 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:47 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:47 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:47 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:47 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:48 smithi189 ceph-mon[32912]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:48 smithi043 ceph-mon[30106]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:48 smithi043 ceph-mon[40255]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:50.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:50 smithi189 ceph-mon[32912]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:50.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:50.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:50.560 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[30106]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[40255]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:50.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:52.235 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.234+0000 7f7ed5ffb640 1 -- 172.21.15.189:0/3182793500 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ed802e8a0 con 0x7f7ec807ad20 2024-09-13T13:27:52.235 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-09-13T13:27:52.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7ec807ad20 msgr2=0x7f7ec807d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:52.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7ec807ad20 0x7f7ec807d1e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0058c0 tx=0x7f7ecc005c60 comp rx=0 tx=0).stop 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8104520 msgr2=0x7f7ed8199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8104520 0x7f7ed8199620 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b970 tx=0x7f7ed000be30 comp rx=0 tx=0).stop 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 shutdown_connections 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f7ec807ad20 0x7f7ec807d1e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8104520 0x7f7ed8199620 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed8103b60 0x7f7ed81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 --2- 172.21.15.189:0/3182793500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ed8102a00 0x7f7ed8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 >> 172.21.15.189:0/3182793500 conn(0x7f7ed80fe2f0 msgr2=0x7f7ed80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 shutdown_connections 2024-09-13T13:27:52.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:52.236+0000 7f7edefa2640 1 -- 172.21.15.189:0/3182793500 wait complete. 2024-09-13T13:27:52.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:52 smithi189 ceph-mon[32912]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:52.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:52 smithi189 ceph-mon[32912]: from='osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T13:27:52.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:52 smithi189 ceph-mon[32912]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T13:27:52.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:52.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:52.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[30106]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[30106]: from='osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[30106]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[40255]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:52.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[40255]: from='osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T13:27:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[40255]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T13:27:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:27:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:52.850 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.5.service 2024-09-13T13:27:52.852 INFO:tasks.cephadm:Deploying osd.6 on smithi189 with /dev/vg_nvme/lv_2... 2024-09-13T13:27:52.852 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-13T13:27:53.425 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:53 smithi189 ceph-mon[32912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T13:27:53.425 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:53 smithi189 ceph-mon[32912]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T13:27:53.426 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:53.426 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:53 smithi189 ceph-mon[32912]: from='osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:53.426 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:53 smithi189 ceph-mon[32912]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:53.662 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:27:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[30106]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[30106]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[30106]: from='osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[30106]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[40255]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[40255]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[40255]: from='osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:53 smithi043 ceph-mon[40255]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:27:54.003 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:27:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:27:53.698+0000 7fea0407a640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T13:27:54.004 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:27:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:27:53.702+0000 7fe9fbe99640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T13:27:54.575 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:54 smithi189 ceph-mon[32912]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:54.575 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:54 smithi189 ceph-mon[32912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:27:54.575 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:54 smithi189 ceph-mon[32912]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T13:27:54.576 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:54.576 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:54 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:54.576 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:54 smithi189 ceph-mon[32912]: from='osd.5 ' entity='osd.5' 2024-09-13T13:27:54.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[30106]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[30106]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[30106]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[30106]: from='osd.5 ' entity='osd.5' 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[40255]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[40255]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[40255]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T13:27:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:54 smithi043 ceph-mon[40255]: from='osd.5 ' entity='osd.5' 2024-09-13T13:27:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:55 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:27:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:55 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:27:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:55 smithi189 ceph-mon[32912]: osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] boot 2024-09-13T13:27:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:55 smithi189 ceph-mon[32912]: osdmap e37: 6 total, 6 up, 6 in 2024-09-13T13:27:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:55 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:55 smithi189 ceph-mon[32912]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[30106]: osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] boot 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[30106]: osdmap e37: 6 total, 6 up, 6 in 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[30106]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[40255]: osd.5 [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] boot 2024-09-13T13:27:55.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[40255]: osdmap e37: 6 total, 6 up, 6 in 2024-09-13T13:27:55.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:27:55.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:55 smithi043 ceph-mon[40255]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T13:27:56.525 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:56 smithi189 ceph-mon[32912]: pgmap v141: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:27:56.526 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:56 smithi189 ceph-mon[32912]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T13:27:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:56 smithi043 ceph-mon[30106]: pgmap v141: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:27:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:56 smithi043 ceph-mon[30106]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T13:27:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:56 smithi043 ceph-mon[40255]: pgmap v141: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:27:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:56 smithi043 ceph-mon[40255]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T13:27:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi189 2024-09-13T13:27:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-13T13:27:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:57 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi189 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi189 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:27:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:57 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:27:58.235 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:27:58.262 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-09-13T13:27:58.511 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:27:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:58 smithi189 ceph-mon[32912]: pgmap v144: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.985+0000 7f9d165a5640 1 -- 172.21.15.189:0/2639354203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 msgr2=0x7f9d10101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.985+0000 7f9d165a5640 1 --2- 172.21.15.189:0/2639354203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d10101ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009cd0 tx=0x7f9d0802a8e0 comp rx=0 tx=0).stop 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 -- 172.21.15.189:0/2639354203 shutdown_connections 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 --2- 172.21.15.189:0/2639354203 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d10103400 0x7f9d10109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 --2- 172.21.15.189:0/2639354203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10102a40 0x7f9d10102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 --2- 172.21.15.189:0/2639354203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d10101ce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 -- 172.21.15.189:0/2639354203 >> 172.21.15.189:0/2639354203 conn(0x7f9d100fd1b0 msgr2=0x7f9d100ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 -- 172.21.15.189:0/2639354203 shutdown_connections 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 -- 172.21.15.189:0/2639354203 wait complete. 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 Processor -- start 2024-09-13T13:27:58.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.986+0000 7f9d165a5640 1 -- start start 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d165a5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d101056a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d165a5640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d10102a40 0x7f9d10103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d165a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10103400 0x7f9d10104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d165a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1010c9b0 con 0x7f9d10103400 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d165a5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1010c830 con 0x7f9d101018e0 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d165a5640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1010cb30 con 0x7f9d10102a40 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d101056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d101056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:56922/0 (socket says 172.21.15.189:56922) 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 -- 172.21.15.189:0/834462574 learned_addr learned my addr 172.21.15.189:0/834462574 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0f7fe640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d10102a40 0x7f9d10103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:58.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d14b42640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10103400 0x7f9d10104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 -- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d10102a40 msgr2=0x7f9d10103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d10102a40 0x7f9d10103f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 -- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10103400 msgr2=0x7f9d10104450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10103400 0x7f9d10104450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 -- 172.21.15.189:0/834462574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d10104990 con 0x7f9d101018e0 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0ffff640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d101056a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d080097b0 tx=0x7f9d08003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0d7fa640 1 -- 172.21.15.189:0/834462574 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080045d0 con 0x7f9d101018e0 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d0d7fa640 1 -- 172.21.15.189:0/834462574 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9d08004770 con 0x7f9d101018e0 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.987+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d10104bc0 con 0x7f9d101018e0 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.988+0000 7f9d0d7fa640 1 -- 172.21.15.189:0/834462574 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080466e0 con 0x7f9d101018e0 2024-09-13T13:27:58.991 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.988+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101a6c20 con 0x7f9d101018e0 2024-09-13T13:27:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:58 smithi043 ceph-mon[30106]: pgmap v144: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:27:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:58 smithi043 ceph-mon[40255]: pgmap v144: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:27:58.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.989+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d10101ce0 con 0x7f9d101018e0 2024-09-13T13:27:58.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.989+0000 7f9d0d7fa640 1 -- 172.21.15.189:0/834462574 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f9d080305a0 con 0x7f9d101018e0 2024-09-13T13:27:58.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.990+0000 7f9d0d7fa640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9cf807ad20 0x7f9cf807d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:27:58.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.990+0000 7f9d0d7fa640 1 -- 172.21.15.189:0/834462574 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4423+0+0 (secure 0 0 0) 0x7f9d0802e030 con 0x7f9d101018e0 2024-09-13T13:27:58.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.990+0000 7f9d0f7fe640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9cf807ad20 0x7f9cf807d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:27:58.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.990+0000 7f9d0f7fe640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9cf807ad20 0x7f9cf807d1e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400abe0 tx=0x7f9d0400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:27:58.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:58.994+0000 7f9d0d7fa640 1 -- 172.21.15.189:0/834462574 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f9d0804b060 con 0x7f9d101018e0 2024-09-13T13:27:59.181 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:27:59.180+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9d1002e710 con 0x7f9cf807ad20 2024-09-13T13:27:59.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:27:59.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:27:59.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:27:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:27:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:27:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:27:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:27:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:27:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:27:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:27:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:00.834 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:00 smithi189 ceph-mon[32912]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:00.834 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:00 smithi189 ceph-mon[32912]: pgmap v145: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:00 smithi043 ceph-mon[30106]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:00 smithi043 ceph-mon[30106]: pgmap v145: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:00 smithi043 ceph-mon[40255]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:00 smithi043 ceph-mon[40255]: pgmap v145: 1 pgs: 1 unknown; 577 KiB data, 538 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:01.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/3313940092' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"} : dispatch 2024-09-13T13:28:01.856 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/3313940092' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"} : dispatch 2024-09-13T13:28:01.856 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"} : dispatch 2024-09-13T13:28:01.856 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"}]': finished 2024-09-13T13:28:01.857 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:01 smithi189 ceph-mon[32912]: osdmap e40: 7 total, 6 up, 7 in 2024-09-13T13:28:01.857 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:01 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"} : dispatch 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"}]': finished 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[30106]: osdmap e40: 7 total, 6 up, 7 in 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/3313940092' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"} : dispatch 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"} : dispatch 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6471db9f-5cfd-413a-8227-befdfb76cb7f"}]': finished 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[40255]: osdmap e40: 7 total, 6 up, 7 in 2024-09-13T13:28:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:01 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:02.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:02 smithi043 ceph-mon[40255]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T13:28:02.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/72886453' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:28:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:02 smithi043 ceph-mon[30106]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T13:28:02.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/72886453' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:28:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:02 smithi189 ceph-mon[32912]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T13:28:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/72886453' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:28:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:04 smithi189 ceph-mon[32912]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-13T13:28:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:04 smithi043 ceph-mon[30106]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-13T13:28:04.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:04 smithi043 ceph-mon[40255]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-13T13:28:06.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:06 smithi189 ceph-mon[32912]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T13:28:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:06 smithi043 ceph-mon[30106]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T13:28:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:06 smithi043 ceph-mon[40255]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T13:28:08.862 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:08 smithi189 ceph-mon[32912]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:28:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:08 smithi043 ceph-mon[30106]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:28:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:08 smithi043 ceph-mon[40255]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:28:09.833 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:09 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-13T13:28:09.833 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:09 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:09 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-13T13:28:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:09 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:09 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-13T13:28:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:09 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:10.877 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:10 smithi189 ceph-mon[32912]: Deploying daemon osd.6 on smithi189 2024-09-13T13:28:10.877 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:10 smithi189 ceph-mon[32912]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:28:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:10 smithi043 ceph-mon[30106]: Deploying daemon osd.6 on smithi189 2024-09-13T13:28:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:10 smithi043 ceph-mon[30106]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:28:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:10 smithi043 ceph-mon[40255]: Deploying daemon osd.6 on smithi189 2024-09-13T13:28:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:10 smithi043 ceph-mon[40255]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T13:28:12.789 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:12 smithi189 ceph-mon[32912]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:12 smithi043 ceph-mon[30106]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:12 smithi043 ceph-mon[40255]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:14 smithi189 ceph-mon[32912]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:14 smithi043 ceph-mon[30106]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:14 smithi043 ceph-mon[40255]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:16 smithi189 ceph-mon[32912]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:16 smithi043 ceph-mon[30106]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:16 smithi043 ceph-mon[40255]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:17 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:17 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:17 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:17.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:17 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:18.091 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:17 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:18.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:17 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:18.092 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:17 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:18.092 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:17 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:18.092 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:17 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:18 smithi189 ceph-mon[32912]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:18 smithi043 ceph-mon[30106]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:18 smithi043 ceph-mon[40255]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[40255]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[30106]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:19 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:19 smithi189 ceph-mon[32912]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:19 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:22.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.290+0000 7f9d0d7fa640 1 -- 172.21.15.189:0/834462574 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9d1002e710 con 0x7f9cf807ad20 2024-09-13T13:28:22.292 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 6 on host 'smithi189' 2024-09-13T13:28:22.294 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9cf807ad20 msgr2=0x7f9cf807d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:22.294 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9cf807ad20 0x7f9cf807d1e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400abe0 tx=0x7f9d0400a2d0 comp rx=0 tx=0).stop 2024-09-13T13:28:22.294 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 msgr2=0x7f9d101056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d101056a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d080097b0 tx=0x7f9d08003f10 comp rx=0 tx=0).stop 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 shutdown_connections 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9cf807ad20 0x7f9cf807d1e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10103400 0x7f9d10104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d10102a40 0x7f9d10103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 --2- 172.21.15.189:0/834462574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d101018e0 0x7f9d101056a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 >> 172.21.15.189:0/834462574 conn(0x7f9d100fd1b0 msgr2=0x7f9d100fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:22.295 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 shutdown_connections 2024-09-13T13:28:22.296 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:22.293+0000 7f9d165a5640 1 -- 172.21.15.189:0/834462574 wait complete. 2024-09-13T13:28:22.623 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:22 smithi189 ceph-mon[32912]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:22.623 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:22 smithi189 ceph-mon[32912]: from='osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T13:28:22.623 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:22 smithi189 ceph-mon[32912]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T13:28:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:22 smithi043 ceph-mon[40255]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:22 smithi043 ceph-mon[40255]: from='osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T13:28:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:22 smithi043 ceph-mon[40255]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T13:28:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:22 smithi043 ceph-mon[30106]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:22 smithi043 ceph-mon[30106]: from='osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T13:28:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:22 smithi043 ceph-mon[30106]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T13:28:22.925 DEBUG:teuthology.orchestra.run.smithi189:osd.6> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.6.service 2024-09-13T13:28:22.928 INFO:tasks.cephadm:Deploying osd.7 on smithi189 with /dev/vg_nvme/lv_1... 2024-09-13T13:28:22.928 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-13T13:28:23.512 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T13:28:23.512 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: osdmap e41: 7 total, 6 up, 7 in 2024-09-13T13:28:23.512 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:23.512 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: from='osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:23.512 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:23.512 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:23.513 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:23.513 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:23.513 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:28:23.267+0000 7f8371d9d640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T13:28:23.513 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:28:23 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:28:23.273+0000 7f836dbc4640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T13:28:23.726 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:28:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: osdmap e41: 7 total, 6 up, 7 in 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: from='osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: osdmap e41: 7 total, 6 up, 7 in 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: from='osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:23.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:23.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:23.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:23.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:23 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:24 smithi189 ceph-mon[32912]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:24 smithi189 ceph-mon[32912]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:28:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:24 smithi189 ceph-mon[32912]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T13:28:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:24 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[30106]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[30106]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:28:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[30106]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T13:28:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[40255]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T13:28:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[40255]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:28:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[40255]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T13:28:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:24 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:25.525 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:25 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:28:25.525 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:25 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:28:25.525 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:25 smithi189 ceph-mon[32912]: osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] boot 2024-09-13T13:28:25.525 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:25 smithi189 ceph-mon[32912]: osdmap e43: 7 total, 7 up, 7 in 2024-09-13T13:28:25.525 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:25 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:25.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[30106]: osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] boot 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[30106]: osdmap e43: 7 total, 7 up, 7 in 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[40255]: osd.6 [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] boot 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[40255]: osdmap e43: 7 total, 7 up, 7 in 2024-09-13T13:28:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:25 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:28:26.584 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:26 smithi189 ceph-mon[32912]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:26.584 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:26 smithi189 ceph-mon[32912]: osdmap e44: 7 total, 7 up, 7 in 2024-09-13T13:28:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:26 smithi043 ceph-mon[30106]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:26 smithi043 ceph-mon[30106]: osdmap e44: 7 total, 7 up, 7 in 2024-09-13T13:28:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:26 smithi043 ceph-mon[40255]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:26 smithi043 ceph-mon[40255]: osdmap e44: 7 total, 7 up, 7 in 2024-09-13T13:28:27.563 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: osdmap e45: 7 total, 7 up, 7 in 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:27.564 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:27 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: osdmap e45: 7 total, 7 up, 7 in 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: osdmap e45: 7 total, 7 up, 7 in 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:27 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:28 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi189 2024-09-13T13:28:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:28 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-13T13:28:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:28 smithi189 ceph-mon[32912]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:28.687 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-13T13:28:28.717 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-09-13T13:28:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:28 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi189 2024-09-13T13:28:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:28 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-13T13:28:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:28 smithi043 ceph-mon[30106]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:28 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi189 2024-09-13T13:28:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:28 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-13T13:28:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:28 smithi043 ceph-mon[40255]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:28.966 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 -- 172.21.15.189:0/35256323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 msgr2=0x7f3a80070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 --2- 172.21.15.189:0/35256323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a80070270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009900 tx=0x7f3a7002a410 comp rx=0 tx=0).stop 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 -- 172.21.15.189:0/35256323 shutdown_connections 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 --2- 172.21.15.189:0/35256323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a800707b0 0x7f3a80071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 --2- 172.21.15.189:0/35256323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a80070270 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 --2- 172.21.15.189:0/35256323 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a8006ec90 0x7f3a8006f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 -- 172.21.15.189:0/35256323 >> 172.21.15.189:0/35256323 conn(0x7f3a8006d580 msgr2=0x7f3a8006d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.421+0000 7f3a87a62640 1 -- 172.21.15.189:0/35256323 shutdown_connections 2024-09-13T13:28:29.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 -- 172.21.15.189:0/35256323 wait complete. 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 Processor -- start 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 -- start start 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a8006ec90 0x7f3a801a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a801a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a800707b0 0x7f3a801a8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8011c9e0 con 0x7f3a8006ec90 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8011c860 con 0x7f3a8006fdf0 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a87a62640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8011cb60 con 0x7f3a800707b0 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a84ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a801a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a84ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a801a78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:43072/0 (socket says 172.21.15.189:43072) 2024-09-13T13:28:29.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.422+0000 7f3a84ffd640 1 -- 172.21.15.189:0/3316191156 learned_addr learned my addr 172.21.15.189:0/3316191156 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a84ffd640 1 -- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a800707b0 msgr2=0x7f3a801a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a85fff640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a800707b0 0x7f3a801a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a857fe640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a8006ec90 0x7f3a801a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a84ffd640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a800707b0 0x7f3a801a8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a84ffd640 1 -- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a8006ec90 msgr2=0x7f3a801a7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a84ffd640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a8006ec90 0x7f3a801a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a84ffd640 1 -- 172.21.15.189:0/3316191156 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a801a8dc0 con 0x7f3a8006fdf0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a84ffd640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a801a78e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006010 tx=0x7f3a700043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a767fc640 1 -- 172.21.15.189:0/3316191156 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7003f070 con 0x7f3a8006fdf0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a767fc640 1 -- 172.21.15.189:0/3316191156 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3a700053d0 con 0x7f3a8006fdf0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a801a9050 con 0x7f3a8006fdf0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a767fc640 1 -- 172.21.15.189:0/3316191156 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70035030 con 0x7f3a8006fdf0 2024-09-13T13:28:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.423+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a801af640 con 0x7f3a8006fdf0 2024-09-13T13:28:29.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.424+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a800702f0 con 0x7f3a8006fdf0 2024-09-13T13:28:29.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.425+0000 7f3a767fc640 1 -- 172.21.15.189:0/3316191156 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f3a70042070 con 0x7f3a8006fdf0 2024-09-13T13:28:29.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.425+0000 7f3a767fc640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f3a6c07ad20 0x7f3a6c07d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:29.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.425+0000 7f3a767fc640 1 -- 172.21.15.189:0/3316191156 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4855+0+0 (secure 0 0 0) 0x7f3a700be510 con 0x7f3a8006fdf0 2024-09-13T13:28:29.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.425+0000 7f3a857fe640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f3a6c07ad20 0x7f3a6c07d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:29.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.426+0000 7f3a857fe640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f3a6c07ad20 0x7f3a6c07d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00a810 tx=0x7f3a7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:29.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.429+0000 7f3a767fc640 1 -- 172.21.15.189:0/3316191156 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3a70085de0 con 0x7f3a8006fdf0 2024-09-13T13:28:29.618 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:29.617+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3a8002e7c0 con 0x7f3a6c07ad20 2024-09-13T13:28:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:30 smithi189 ceph-mon[32912]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:30 smithi189 ceph-mon[32912]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:28:30.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:28:30.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:30 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[30106]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[30106]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[40255]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[40255]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T13:28:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:30 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:32.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[30106]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:32.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/2604451584' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"} : dispatch 2024-09-13T13:28:32.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"} : dispatch 2024-09-13T13:28:32.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"}]': finished 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[30106]: osdmap e46: 8 total, 7 up, 8 in 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[40255]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/2604451584' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"} : dispatch 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"} : dispatch 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"}]': finished 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[40255]: osdmap e46: 8 total, 7 up, 8 in 2024-09-13T13:28:32.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:32 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:32.565 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:32 smithi189 ceph-mon[32912]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:32.565 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/2604451584' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"} : dispatch 2024-09-13T13:28:32.565 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"} : dispatch 2024-09-13T13:28:32.565 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5989d1a-268c-46f0-8e74-6f8b30cb0d19"}]': finished 2024-09-13T13:28:32.565 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:32 smithi189 ceph-mon[32912]: osdmap e46: 8 total, 7 up, 8 in 2024-09-13T13:28:32.565 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:32 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/622714239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:28:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/622714239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:28:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/622714239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T13:28:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:34 smithi189 ceph-mon[32912]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:34.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:34 smithi043 ceph-mon[30106]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:34.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:34 smithi043 ceph-mon[40255]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:36.493 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:36 smithi189 ceph-mon[32912]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:36.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:36 smithi043 ceph-mon[30106]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:36 smithi043 ceph-mon[40255]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:38.485 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:38 smithi189 ceph-mon[32912]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:38 smithi043 ceph-mon[30106]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:38 smithi043 ceph-mon[40255]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:39 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-13T13:28:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:39 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:39 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-13T13:28:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:39 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:39 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-13T13:28:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:39 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:40.552 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:40 smithi189 ceph-mon[32912]: Deploying daemon osd.7 on smithi189 2024-09-13T13:28:40.552 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:40 smithi189 ceph-mon[32912]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:40 smithi043 ceph-mon[30106]: Deploying daemon osd.7 on smithi189 2024-09-13T13:28:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:40 smithi043 ceph-mon[30106]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:40 smithi043 ceph-mon[40255]: Deploying daemon osd.7 on smithi189 2024-09-13T13:28:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:40 smithi043 ceph-mon[40255]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:42 smithi189 ceph-mon[32912]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:42 smithi043 ceph-mon[30106]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:42.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:42 smithi043 ceph-mon[40255]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:44.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:44 smithi189 ceph-mon[32912]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:44 smithi043 ceph-mon[30106]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:44 smithi043 ceph-mon[40255]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:46 smithi189 ceph-mon[32912]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:46 smithi043 ceph-mon[30106]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:46 smithi043 ceph-mon[40255]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:47 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:47 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:47 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:47 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:47 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:47 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:47 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:48 smithi189 ceph-mon[32912]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:48 smithi043 ceph-mon[30106]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:48 smithi043 ceph-mon[40255]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[30106]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[40255]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:50 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:50 smithi189 ceph-mon[32912]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:50.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:50.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:50.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:50.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:50 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:51.637 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:51 smithi189 ceph-mon[32912]: from='osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T13:28:51.637 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:51 smithi189 ceph-mon[32912]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T13:28:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:51 smithi043 ceph-mon[30106]: from='osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T13:28:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:51 smithi043 ceph-mon[30106]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T13:28:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:51 smithi043 ceph-mon[40255]: from='osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T13:28:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:51 smithi043 ceph-mon[40255]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T13:28:52.289 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 7 on host 'smithi189' 2024-09-13T13:28:52.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.288+0000 7f3a767fc640 1 -- 172.21.15.189:0/3316191156 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a8002e7c0 con 0x7f3a6c07ad20 2024-09-13T13:28:52.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f3a6c07ad20 msgr2=0x7f3a6c07d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:52.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f3a6c07ad20 0x7f3a6c07d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00a810 tx=0x7f3a7c008040 comp rx=0 tx=0).stop 2024-09-13T13:28:52.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 msgr2=0x7f3a801a78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:52.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a801a78e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006010 tx=0x7f3a700043b0 comp rx=0 tx=0).stop 2024-09-13T13:28:52.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 shutdown_connections 2024-09-13T13:28:52.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f3a6c07ad20 0x7f3a6c07d1e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:52.292 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a800707b0 0x7f3a801a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:52.292 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8006fdf0 0x7f3a801a78e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:52.292 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 --2- 172.21.15.189:0/3316191156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a8006ec90 0x7f3a801a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:52.292 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 >> 172.21.15.189:0/3316191156 conn(0x7f3a8006d580 msgr2=0x7f3a8010be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:52.292 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 shutdown_connections 2024-09-13T13:28:52.292 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:52.290+0000 7f3a87a62640 1 -- 172.21.15.189:0/3316191156 wait complete. 2024-09-13T13:28:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T13:28:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: osdmap e47: 8 total, 7 up, 8 in 2024-09-13T13:28:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: from='osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:52 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: osdmap e47: 8 total, 7 up, 8 in 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: from='osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: osdmap e47: 8 total, 7 up, 8 in 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: from='osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]} : dispatch 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:52 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:52.983 DEBUG:teuthology.orchestra.run.smithi189:osd.7> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.7.service 2024-09-13T13:28:52.985 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-13T13:28:52.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T13:28:53.243 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: from='osd.7 ' entity='osd.7' 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] boot 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: osdmap e49: 8 total, 8 up, 8 in 2024-09-13T13:28:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:53 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:28:53.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T13:28:53.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: from='osd.7 ' entity='osd.7' 2024-09-13T13:28:53.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] boot 2024-09-13T13:28:53.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: osdmap e49: 8 total, 8 up, 8 in 2024-09-13T13:28:53.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-13T13:28:53.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T13:28:53.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: from='osd.7 ' entity='osd.7' 2024-09-13T13:28:53.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: osd.7 [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] boot 2024-09-13T13:28:53.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: osdmap e49: 8 total, 8 up, 8 in 2024-09-13T13:28:53.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:53 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 -- 172.21.15.43:0/412279759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec101920 msgr2=0x7f61ec101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/412279759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec101920 0x7f61ec101d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009960 tx=0x7f61dc02a460 comp rx=0 tx=0).stop 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 -- 172.21.15.43:0/412279759 shutdown_connections 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/412279759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ec103440 0x7f61ec109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/412279759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61ec102a80 0x7f61ec102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/412279759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec101920 0x7f61ec101d20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 -- 172.21.15.43:0/412279759 >> 172.21.15.43:0/412279759 conn(0x7f61ec0fd1d0 msgr2=0x7f61ec0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 -- 172.21.15.43:0/412279759 shutdown_connections 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 -- 172.21.15.43:0/412279759 wait complete. 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 Processor -- start 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.800+0000 7f61f0d9b640 1 -- start start 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61f0d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ec101920 0x7f61ec198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61f0d9b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec102a80 0x7f61ec198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61f0d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61ec103440 0x7f61ec1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61f0d9b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10ca10 con 0x7f61ec101920 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61f0d9b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10c890 con 0x7f61ec103440 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61f0d9b640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f61ec10cb90 con 0x7f61ec102a80 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec102a80 0x7f61ec198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ec101920 0x7f61ec198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec102a80 0x7f61ec198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54788/0 (socket says 172.21.15.43:54788) 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 -- 172.21.15.43:0/1240456203 learned_addr learned my addr 172.21.15.43:0/1240456203 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61ead9d640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61ec103440 0x7f61ec1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 -- 172.21.15.43:0/1240456203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61ec103440 msgr2=0x7f61ec1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61ec103440 0x7f61ec1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 -- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ec101920 msgr2=0x7f61ec198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ec101920 0x7f61ec198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 -- 172.21.15.43:0/1240456203 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec199f80 con 0x7f61ec102a80 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61ead9d640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61ec103440 0x7f61ec1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61ea59c640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ec101920 0x7f61ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61e9d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec102a80 0x7f61ec198aa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c970 tx=0x7f61e000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.801+0000 7f61db7fe640 1 -- 172.21.15.43:0/1240456203 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0007bb0 con 0x7f61ec102a80 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.802+0000 7f61db7fe640 1 -- 172.21.15.43:0/1240456203 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f61e0007d50 con 0x7f61ec102a80 2024-09-13T13:28:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.802+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec19a270 con 0x7f61ec102a80 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.802+0000 7f61db7fe640 1 -- 172.21.15.43:0/1240456203 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0004c90 con 0x7f61ec102a80 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.802+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec1a0950 con 0x7f61ec102a80 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.803+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ec101d20 con 0x7f61ec102a80 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.803+0000 7f61db7fe640 1 -- 172.21.15.43:0/1240456203 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f61e0004eb0 con 0x7f61ec102a80 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.804+0000 7f61db7fe640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f61d407ad20 0x7f61d407d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.804+0000 7f61db7fe640 1 -- 172.21.15.43:0/1240456203 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 5320+0+0 (secure 0 0 0) 0x7f61e009d6e0 con 0x7f61ec102a80 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.804+0000 7f61ea59c640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f61d407ad20 0x7f61d407d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.804+0000 7f61ea59c640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f61d407ad20 0x7f61d407d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c00 tx=0x7f61dc005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.808+0000 7f61db7fe640 1 -- 172.21.15.43:0/1240456203 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f61e0014030 con 0x7f61ec102a80 2024-09-13T13:28:53.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.996+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f61ec02e810 con 0x7f61ec102a80 2024-09-13T13:28:53.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.997+0000 7f61db7fe640 1 -- 172.21.15.43:0/1240456203 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v49) ==== 74+0+130 (secure 0 0 0) 0x7f61e0069240 con 0x7f61ec102a80 2024-09-13T13:28:53.998 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f61d407ad20 msgr2=0x7f61d407d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f61d407ad20 0x7f61d407d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c00 tx=0x7f61dc005d50 comp rx=0 tx=0).stop 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec102a80 msgr2=0x7f61ec198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec102a80 0x7f61ec198aa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c970 tx=0x7f61e000ce30 comp rx=0 tx=0).stop 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 shutdown_connections 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f61d407ad20 0x7f61d407d1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61ec103440 0x7f61ec1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61ec102a80 0x7f61ec198aa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 --2- 172.21.15.43:0/1240456203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ec101920 0x7f61ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 >> 172.21.15.43:0/1240456203 conn(0x7f61ec0fd1d0 msgr2=0x7f61ec0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 shutdown_connections 2024-09-13T13:28:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:53.999+0000 7f61f0d9b640 1 -- 172.21.15.43:0/1240456203 wait complete. 2024-09-13T13:28:54.649 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1726234133,"num_in_osds":8,"osd_in_since":1726234111,"num_remapped_pgs":1} 2024-09-13T13:28:54.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T13:28:54.907 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:28:55.140 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[40255]: purged_snaps scrub starts 2024-09-13T13:28:55.140 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[40255]: purged_snaps scrub ok 2024-09-13T13:28:55.140 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[40255]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:55.140 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1240456203' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T13:28:55.140 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[40255]: osdmap e50: 8 total, 8 up, 8 in 2024-09-13T13:28:55.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[30106]: purged_snaps scrub starts 2024-09-13T13:28:55.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[30106]: purged_snaps scrub ok 2024-09-13T13:28:55.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[30106]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:55.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1240456203' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T13:28:55.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:54 smithi043 ceph-mon[30106]: osdmap e50: 8 total, 8 up, 8 in 2024-09-13T13:28:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:54 smithi189 ceph-mon[32912]: purged_snaps scrub starts 2024-09-13T13:28:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:54 smithi189 ceph-mon[32912]: purged_snaps scrub ok 2024-09-13T13:28:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:54 smithi189 ceph-mon[32912]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T13:28:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1240456203' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T13:28:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:54 smithi189 ceph-mon[32912]: osdmap e50: 8 total, 8 up, 8 in 2024-09-13T13:28:55.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 -- 172.21.15.43:0/3976628400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128108030 msgr2=0x7fd12810e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/3976628400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128108030 0x7fd12810e830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd120009be0 tx=0x7fd12002a6c0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 -- 172.21.15.43:0/3976628400 shutdown_connections 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/3976628400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128108030 0x7fd12810e830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/3976628400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 0x7fd128107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/3976628400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd128101ca0 0x7fd128102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 -- 172.21.15.43:0/3976628400 >> 172.21.15.43:0/3976628400 conn(0x7fd1280fd710 msgr2=0x7fd1280ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 -- 172.21.15.43:0/3976628400 shutdown_connections 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.362+0000 7fd12ed1d640 1 -- 172.21.15.43:0/3976628400 wait complete. 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 Processor -- start 2024-09-13T13:28:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 -- start start 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128101ca0 0x7fd1281987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 0x7fd128198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd128108030 0x7fd12819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281111c0 con 0x7fd128101ca0 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd128111040 con 0x7fd128108030 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12ed1d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fd128111340 con 0x7fd1281076b0 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 0x7fd128198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 0x7fd128198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54796/0 (socket says 172.21.15.43:54796) 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 -- 172.21.15.43:0/2502040735 learned_addr learned my addr 172.21.15.43:0/2502040735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12cab9640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128101ca0 0x7fd1281987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12d2ba640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd128108030 0x7fd12819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 -- 172.21.15.43:0/2502040735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd128108030 msgr2=0x7fd12819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd128108030 0x7fd12819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 -- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128101ca0 msgr2=0x7fd1281987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128101ca0 0x7fd1281987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd127fff640 1 -- 172.21.15.43:0/2502040735 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12819e880 con 0x7fd1281076b0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.363+0000 7fd12cab9640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128101ca0 0x7fd1281987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.364+0000 7fd127fff640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 0x7fd128198d10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd11000e9c0 tx=0x7fd11000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.364+0000 7fd125ffb640 1 -- 172.21.15.43:0/2502040735 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11000cde0 con 0x7fd1281076b0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.364+0000 7fd125ffb640 1 -- 172.21.15.43:0/2502040735 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fd110004470 con 0x7fd1281076b0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.364+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12819eb70 con 0x7fd1281076b0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.364+0000 7fd125ffb640 1 -- 172.21.15.43:0/2502040735 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110003e50 con 0x7fd1281076b0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.364+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a5230 con 0x7fd1281076b0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.365+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd128102080 con 0x7fd1281076b0 2024-09-13T13:28:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.366+0000 7fd125ffb640 1 -- 172.21.15.43:0/2502040735 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7fd110004000 con 0x7fd1281076b0 2024-09-13T13:28:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.366+0000 7fd125ffb640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd11407aec0 0x7fd11407d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.366+0000 7fd125ffb640 1 -- 172.21.15.43:0/2502040735 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5287+0+0 (secure 0 0 0) 0x7fd11009ddf0 con 0x7fd1281076b0 2024-09-13T13:28:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.366+0000 7fd12cab9640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd11407aec0 0x7fd11407d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.367+0000 7fd12cab9640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd11407aec0 0x7fd11407d380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1180058c0 tx=0x7fd11800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.371+0000 7fd125ffb640 1 -- 172.21.15.43:0/2502040735 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fd110065510 con 0x7fd1281076b0 2024-09-13T13:28:55.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.558+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd12810ae20 con 0x7fd1281076b0 2024-09-13T13:28:55.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.559+0000 7fd125ffb640 1 -- 172.21.15.43:0/2502040735 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13575 (secure 0 0 0) 0x7fd1100698f0 con 0x7fd1281076b0 2024-09-13T13:28:55.560 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:28:55.561 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":51,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","created":"2024-09-13T13:23:11.853754+0000","modified":"2024-09-13T13:28:55.437603+0000","last_up_change":"2024-09-13T13:28:53.434914+0000","last_in_change":"2024-09-13T13:28:31.961750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T13:26:27.611335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2bd16e07-e057-4c72-83d3-17490e876e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6803","nonce":2675188630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6805","nonce":2675188630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6809","nonce":2675188630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6807","nonce":2675188630}]},"public_addr":"172.21.15.43:6803/2675188630","cluster_addr":"172.21.15.43:6805/2675188630","heartbeat_back_addr":"172.21.15.43:6809/2675188630","heartbeat_front_addr":"172.21.15.43:6807/2675188630","state":["exists","up"]},{"osd":1,"uuid":"950ef5d7-8755-4a89-b1b3-953dd7f3bd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6811","nonce":966353703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6813","nonce":966353703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6817","nonce":966353703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6815","nonce":966353703}]},"public_addr":"172.21.15.43:6811/966353703","cluster_addr":"172.21.15.43:6813/966353703","heartbeat_back_addr":"172.21.15.43:6817/966353703","heartbeat_front_addr":"172.21.15.43:6815/966353703","state":["exists","up"]},{"osd":2,"uuid":"8cad8dbd-500f-46d9-9f70-27d4b12c8835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6819","nonce":601321569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6821","nonce":601321569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6825","nonce":601321569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6823","nonce":601321569}]},"public_addr":"172.21.15.43:6819/601321569","cluster_addr":"172.21.15.43:6821/601321569","heartbeat_back_addr":"172.21.15.43:6825/601321569","heartbeat_front_addr":"172.21.15.43:6823/601321569","state":["exists","up"]},{"osd":3,"uuid":"c24c101f-2a35-4ed4-8cf3-6ec3824a78de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6827","nonce":3529572742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6829","nonce":3529572742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6833","nonce":3529572742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6831","nonce":3529572742}]},"public_addr":"172.21.15.43:6827/3529572742","cluster_addr":"172.21.15.43:6829/3529572742","heartbeat_back_addr":"172.21.15.43:6833/3529572742","heartbeat_front_addr":"172.21.15.43:6831/3529572742","state":["exists","up"]},{"osd":4,"uuid":"6d500438-0ae5-4c98-bd0b-ed0955ef9aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6801","nonce":1782399195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6803","nonce":1782399195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6807","nonce":1782399195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6805","nonce":1782399195}]},"public_addr":"172.21.15.189:6801/1782399195","cluster_addr":"172.21.15.189:6803/1782399195","heartbeat_back_addr":"172.21.15.189:6807/1782399195","heartbeat_front_addr":"172.21.15.189:6805/1782399195","state":["exists","up"]},{"osd":5,"uuid":"141633af-2b7a-4224-8c9e-43d82589627b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6809","nonce":1660315521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6811","nonce":1660315521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6815","nonce":1660315521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6813","nonce":1660315521}]},"public_addr":"172.21.15.189:6809/1660315521","cluster_addr":"172.21.15.189:6811/1660315521","heartbeat_back_addr":"172.21.15.189:6815/1660315521","heartbeat_front_addr":"172.21.15.189:6813/1660315521","state":["exists","up"]},{"osd":6,"uuid":"6471db9f-5cfd-413a-8227-befdfb76cb7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6817","nonce":1554583783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6819","nonce":1554583783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6823","nonce":1554583783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6821","nonce":1554583783}]},"public_addr":"172.21.15.189:6817/1554583783","cluster_addr":"172.21.15.189:6819/1554583783","heartbeat_back_addr":"172.21.15.189:6823/1554583783","heartbeat_front_addr":"172.21.15.189:6821/1554583783","state":["exists","up"]},{"osd":7,"uuid":"b5989d1a-268c-46f0-8e74-6f8b30cb0d19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6825","nonce":1994869283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6827","nonce":1994869283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6831","nonce":1994869283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6829","nonce":1994869283}]},"public_addr":"172.21.15.189:6825/1994869283","cluster_addr":"172.21.15.189:6827/1994869283","heartbeat_back_addr":"172.21.15.189:6831/1994869283","heartbeat_front_addr":"172.21.15.189:6829/1994869283","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:25:26.648165+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:25:55.036841+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:26:25.392931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:26:54.628347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:27:22.366719+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:27:52.620989+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:28:22.633493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:28:52.273337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/85481396":"2024-09-14T13:24:07.149596+0000","172.21.15.43:6800/85481396":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/3626241883":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/1182815574":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/101460694":"2024-09-14T13:23:33.399872+0000","172.21.15.43:0/2407577026":"2024-09-14T13:23:33.399872+0000","172.21.15.43:0/1023127578":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/3370980276":"2024-09-14T13:23:33.399872+0000","172.21.15.43:6801/3041194058":"2024-09-14T13:23:33.399872+0000","172.21.15.43:6800/3041194058":"2024-09-14T13:23:33.399872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.561+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd11407aec0 msgr2=0x7fd11407d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.561+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd11407aec0 0x7fd11407d380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1180058c0 tx=0x7fd11800a2d0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.561+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 msgr2=0x7fd128198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.561+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 0x7fd128198d10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd11000e9c0 tx=0x7fd11000ee80 comp rx=0 tx=0).stop 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 shutdown_connections 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd11407aec0 0x7fd11407d380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd128108030 0x7fd12819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1281076b0 0x7fd128198d10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 --2- 172.21.15.43:0/2502040735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128101ca0 0x7fd1281987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 >> 172.21.15.43:0/2502040735 conn(0x7fd1280fd710 msgr2=0x7fd12806b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 shutdown_connections 2024-09-13T13:28:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:55.562+0000 7fd12ed1d640 1 -- 172.21.15.43:0/2502040735 wait complete. 2024-09-13T13:28:56.171 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T13:26:27.611335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T13:28:56.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-13T13:28:56.425 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:28:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:28:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T13:28:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2502040735' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:28:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: Detected new or changed devices on smithi189 2024-09-13T13:28:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[30106]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2502040735' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: Detected new or changed devices on smithi189 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-13T13:28:56.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:56.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:56.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:56 smithi043 ceph-mon[40255]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-13T13:28:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 -- 172.21.15.43:0/23545422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00109130 msgr2=0x7fbe0010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 --2- 172.21.15.43:0/23545422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00109130 0x7fbe0010f930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009be0 tx=0x7fbdf802a6c0 comp rx=0 tx=0).stop 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 -- 172.21.15.43:0/23545422 shutdown_connections 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 --2- 172.21.15.43:0/23545422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00109130 0x7fbe0010f930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 --2- 172.21.15.43:0/23545422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe001087b0 0x7fbe00108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 --2- 172.21.15.43:0/23545422 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe00102da0 0x7fbe00103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 -- 172.21.15.43:0/23545422 >> 172.21.15.43:0/23545422 conn(0x7fbe000fe810 msgr2=0x7fbe00100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 -- 172.21.15.43:0/23545422 shutdown_connections 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 -- 172.21.15.43:0/23545422 wait complete. 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.848+0000 7fbe05db6640 1 Processor -- start 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbe05db6640 1 -- start start 2024-09-13T13:28:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbe05db6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00102da0 0x7fbe00198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbe05db6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe001087b0 0x7fbe00198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbe05db6640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe00109130 0x7fbe0019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbe05db6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe00112280 con 0x7fbe00102da0 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbe05db6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbe00112100 con 0x7fbe001087b0 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbe05db6640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fbe00112400 con 0x7fbe00109130 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00102da0 0x7fbe00198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00102da0 0x7fbe00198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59824/0 (socket says 172.21.15.43:59824) 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 -- 172.21.15.43:0/3576241843 learned_addr learned my addr 172.21.15.43:0/3576241843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdfffff640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe00109130 0x7fbe0019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdfeffd640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe001087b0 0x7fbe00198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 -- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe00109130 msgr2=0x7fbe0019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe00109130 0x7fbe0019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 -- 172.21.15.43:0/3576241843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe001087b0 msgr2=0x7fbe00198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe001087b0 0x7fbe00198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdff7fe640 1 -- 172.21.15.43:0/3576241843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0019e820 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.849+0000 7fbdfeffd640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe001087b0 0x7fbe00198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.850+0000 7fbdff7fe640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00102da0 0x7fbe00198770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000e9c0 tx=0x7fbdf000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.850+0000 7fbdfcff9640 1 -- 172.21.15.43:0/3576241843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000cde0 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.850+0000 7fbdfcff9640 1 -- 172.21.15.43:0/3576241843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fbdf0004470 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.850+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0019eb10 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.850+0000 7fbdfcff9640 1 -- 172.21.15.43:0/3576241843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0003e50 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.850+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe001a53a0 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.851+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe00103180 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.852+0000 7fbdfcff9640 1 -- 172.21.15.43:0/3576241843 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7fbdf0004000 con 0x7fbe00102da0 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.852+0000 7fbdfcff9640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fbde807aec0 0x7fbde807d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.852+0000 7fbdfeffd640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fbde807aec0 0x7fbde807d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:56.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.852+0000 7fbdfcff9640 1 -- 172.21.15.43:0/3576241843 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5287+0+0 (secure 0 0 0) 0x7fbdf009ddd0 con 0x7fbe00102da0 2024-09-13T13:28:56.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.853+0000 7fbdfeffd640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fbde807aec0 0x7fbde807d380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe000ffba0 tx=0x7fbdf4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:56.857+0000 7fbdfcff9640 1 -- 172.21.15.43:0/3576241843 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fbdf0065570 con 0x7fbe00102da0 2024-09-13T13:28:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:28:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T13:28:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2502040735' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:28:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: Detected new or changed devices on smithi189 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:56.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:56 smithi189 ceph-mon[32912]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-13T13:28:57.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.039+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe0010af10 con 0x7fbe00102da0 2024-09-13T13:28:57.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.039+0000 7fbdfcff9640 1 -- 172.21.15.43:0/3576241843 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7fbdf0069950 con 0x7fbe00102da0 2024-09-13T13:28:57.040 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-13T13:28:57.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fbde807aec0 msgr2=0x7fbde807d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fbde807aec0 0x7fbde807d380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe000ffba0 tx=0x7fbdf4005d30 comp rx=0 tx=0).stop 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00102da0 msgr2=0x7fbe00198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00102da0 0x7fbe00198770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000e9c0 tx=0x7fbdf000ee80 comp rx=0 tx=0).stop 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 shutdown_connections 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fbde807aec0 0x7fbde807d380 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe00109130 0x7fbe0019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe001087b0 0x7fbe00198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 --2- 172.21.15.43:0/3576241843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00102da0 0x7fbe00198770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 >> 172.21.15.43:0/3576241843 conn(0x7fbe000fe810 msgr2=0x7fbe00100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 shutdown_connections 2024-09-13T13:28:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:28:57.041+0000 7fbe05db6640 1 -- 172.21.15.43:0/3576241843 wait complete. 2024-09-13T13:28:57.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3576241843' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-13T13:28:57.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3576241843' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-13T13:28:57.729 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi043 2024-09-13T13:28:57.729 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi043=foo.a' 2024-09-13T13:28:57.758 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3576241843' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-13T13:28:57.991 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:28:58.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 -- 172.21.15.189:0/4162271319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601086f0 msgr2=0x7fdd6010abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:58.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/4162271319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601086f0 0x7fdd6010abb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009900 tx=0x7fdd4802a410 comp rx=0 tx=0).stop 2024-09-13T13:28:58.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 -- 172.21.15.189:0/4162271319 shutdown_connections 2024-09-13T13:28:58.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/4162271319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601086f0 0x7fdd6010abb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/4162271319 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd60105b60 0x7fdd60107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/4162271319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd6006b650 0x7fdd60105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 -- 172.21.15.189:0/4162271319 >> 172.21.15.189:0/4162271319 conn(0x7fdd600fd1b0 msgr2=0x7fdd600ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 -- 172.21.15.189:0/4162271319 shutdown_connections 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.434+0000 7fdd66ae7640 1 -- 172.21.15.189:0/4162271319 wait complete. 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 Processor -- start 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 -- start start 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd6006b650 0x7fdd601982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd60105b60 0x7fdd60198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd601086f0 0x7fdd60199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6010dad0 con 0x7fdd601086f0 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6010d950 con 0x7fdd6006b650 2024-09-13T13:28:58.467 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd66ae7640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6010dc50 con 0x7fdd60105b60 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd6006b650 0x7fdd601982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd6006b650 0x7fdd601982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:46912/0 (socket says 172.21.15.189:46912) 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 -- 172.21.15.189:0/2219896022 learned_addr learned my addr 172.21.15.189:0/2219896022 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd5ffff640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd60105b60 0x7fdd60198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd65084640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd601086f0 0x7fdd60199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 -- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd60105b60 msgr2=0x7fdd60198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd60105b60 0x7fdd60198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 -- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd601086f0 msgr2=0x7fdd60199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd601086f0 0x7fdd60199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 -- 172.21.15.189:0/2219896022 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd60199de0 con 0x7fdd6006b650 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd5ffff640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd60105b60 0x7fdd60198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:28:58.468 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.435+0000 7fdd64883640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd6006b650 0x7fdd601982b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d950 tx=0x7fdd5800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.436+0000 7fdd5dffb640 1 -- 172.21.15.189:0/2219896022 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58017030 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.436+0000 7fdd5dffb640 1 -- 172.21.15.189:0/2219896022 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fdd58004020 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.436+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6019a0d0 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.436+0000 7fdd5dffb640 1 -- 172.21.15.189:0/2219896022 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58005430 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.436+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd601a07b0 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.437+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd60105fa0 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.437+0000 7fdd5dffb640 1 -- 172.21.15.189:0/2219896022 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7fdd58009a30 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.438+0000 7fdd5dffb640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fdd4c07ad20 0x7fdd4c07d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.438+0000 7fdd5dffb640 1 -- 172.21.15.189:0/2219896022 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5287+0+0 (secure 0 0 0) 0x7fdd5809d580 con 0x7fdd6006b650 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.438+0000 7fdd5ffff640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fdd4c07ad20 0x7fdd4c07d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.439+0000 7fdd5ffff640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fdd4c07ad20 0x7fdd4c07d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd500092f0 tx=0x7fdd50009280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:28:58.469 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.442+0000 7fdd5dffb640 1 -- 172.21.15.189:0/2219896022 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fdd58064d20 con 0x7fdd6006b650 2024-09-13T13:28:58.634 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.632+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}) -- 0x7fdd60063590 con 0x7fdd4c07ad20 2024-09-13T13:28:58.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.639+0000 7fdd5dffb640 1 -- 172.21.15.189:0/2219896022 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fdd60063590 con 0x7fdd4c07ad20 2024-09-13T13:28:58.641 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled rgw.foo.a update... 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.641+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fdd4c07ad20 msgr2=0x7fdd4c07d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.641+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fdd4c07ad20 0x7fdd4c07d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd500092f0 tx=0x7fdd50009280 comp rx=0 tx=0).stop 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.641+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd6006b650 msgr2=0x7fdd601982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.641+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd6006b650 0x7fdd601982b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d950 tx=0x7fdd5800de10 comp rx=0 tx=0).stop 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 shutdown_connections 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fdd4c07ad20 0x7fdd4c07d1e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd601086f0 0x7fdd60199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd60105b60 0x7fdd60198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 --2- 172.21.15.189:0/2219896022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd6006b650 0x7fdd601982b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 >> 172.21.15.189:0/2219896022 conn(0x7fdd600fd1b0 msgr2=0x7fdd60103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 shutdown_connections 2024-09-13T13:28:58.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:28:58.642+0000 7fdd66ae7640 1 -- 172.21.15.189:0/2219896022 wait complete. 2024-09-13T13:28:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:58 smithi043 ceph-mon[30106]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:28:58.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:58 smithi043 ceph-mon[40255]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:28:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:58 smithi189 ceph-mon[32912]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:28:59.313 DEBUG:teuthology.orchestra.run.smithi043:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-13T13:28:59.315 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi189 2024-09-13T13:28:59.315 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-13T13:28:59.600 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:28:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-13T13:28:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-13T13:28:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-13T13:28:59.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:59.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:28:59 smithi189 ceph-mon[32912]: Deploying daemon rgw.foo.a on smithi043 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-13T13:28:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[30106]: Deploying daemon rgw.foo.a on smithi043 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-13T13:28:59.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-13T13:28:59.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:28:59.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:28:59.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:28:59 smithi043 ceph-mon[40255]: Deploying daemon rgw.foo.a on smithi043 2024-09-13T13:29:00.044 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 -- 172.21.15.189:0/404687489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c102a00 msgr2=0x7fc14c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:00.044 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 --2- 172.21.15.189:0/404687489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c102a00 0x7fc14c102e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc148009930 tx=0x7fc14802a420 comp rx=0 tx=0).stop 2024-09-13T13:29:00.044 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 -- 172.21.15.189:0/404687489 shutdown_connections 2024-09-13T13:29:00.044 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 --2- 172.21.15.189:0/404687489 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14c104520 0x7fc14c10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.044 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 --2- 172.21.15.189:0/404687489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14c103b60 0x7fc14c103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 --2- 172.21.15.189:0/404687489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c102a00 0x7fc14c102e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 -- 172.21.15.189:0/404687489 >> 172.21.15.189:0/404687489 conn(0x7fc14c0fe2f0 msgr2=0x7fc14c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 -- 172.21.15.189:0/404687489 shutdown_connections 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.040+0000 7fc15353b640 1 -- 172.21.15.189:0/404687489 wait complete. 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 Processor -- start 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 -- start start 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14c102a00 0x7fc14c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c103b60 0x7fc14c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14c104520 0x7fc14c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c10dad0 con 0x7fc14c104520 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c10d950 con 0x7fc14c103b60 2024-09-13T13:29:00.045 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc15353b640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fc14c10dc50 con 0x7fc14c102a00 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c103b60 0x7fc14c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c103b60 0x7fc14c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:46932/0 (socket says 172.21.15.189:46932) 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 -- 172.21.15.189:0/2105633564 learned_addr learned my addr 172.21.15.189:0/2105633564 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc1512d7640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14c102a00 0x7fc14c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc151ad8640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14c104520 0x7fc14c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 -- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14c102a00 msgr2=0x7fc14c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14c102a00 0x7fc14c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 -- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14c104520 msgr2=0x7fc14c199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14c104520 0x7fc14c199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc150ad6640 1 -- 172.21.15.189:0/2105633564 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c199dd0 con 0x7fc14c103b60 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.041+0000 7fc1512d7640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14c102a00 0x7fc14c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:29:00.046 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.042+0000 7fc150ad6640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c103b60 0x7fc14c1988f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00c920 tx=0x7fc13c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:00.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.042+0000 7fc1427fc640 1 -- 172.21.15.189:0/2105633564 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c007c70 con 0x7fc14c103b60 2024-09-13T13:29:00.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.042+0000 7fc1427fc640 1 -- 172.21.15.189:0/2105633564 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc13c007e10 con 0x7fc14c103b60 2024-09-13T13:29:00.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.042+0000 7fc1427fc640 1 -- 172.21.15.189:0/2105633564 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c018660 con 0x7fc14c103b60 2024-09-13T13:29:00.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.042+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14c19a0c0 con 0x7fc14c103b60 2024-09-13T13:29:00.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.042+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14c1a07a0 con 0x7fc14c103b60 2024-09-13T13:29:00.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.043+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc14c102e00 con 0x7fc14c103b60 2024-09-13T13:29:00.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.043+0000 7fc1427fc640 1 -- 172.21.15.189:0/2105633564 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7fc13c00f450 con 0x7fc14c103b60 2024-09-13T13:29:00.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.044+0000 7fc1427fc640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc13807ad20 0x7fc13807d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:00.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.044+0000 7fc1427fc640 1 -- 172.21.15.189:0/2105633564 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5287+0+0 (secure 0 0 0) 0x7fc13c09db20 con 0x7fc14c103b60 2024-09-13T13:29:00.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.044+0000 7fc1512d7640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc13807ad20 0x7fc13807d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:00.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.045+0000 7fc1512d7640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc13807ad20 0x7fc13807d1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c00 tx=0x7fc148005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:00.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.048+0000 7fc1427fc640 1 -- 172.21.15.189:0/2105633564 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc13c014030 con 0x7fc14c103b60 2024-09-13T13:29:00.226 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.224+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7fc14c063d50 con 0x7fc14c103b60 2024-09-13T13:29:00.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.660+0000 7fc1427fc640 1 -- 172.21.15.189:0/2105633564 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v52) ==== 160+0+0 (secure 0 0 0) 0x7fc13c0652c0 con 0x7fc14c103b60 2024-09-13T13:29:00.662 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'datapool' created 2024-09-13T13:29:00.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc13807ad20 msgr2=0x7fc13807d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:00.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc13807ad20 0x7fc13807d1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c00 tx=0x7fc148005a20 comp rx=0 tx=0).stop 2024-09-13T13:29:00.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c103b60 msgr2=0x7fc14c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:00.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c103b60 0x7fc14c1988f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00c920 tx=0x7fc13c00cde0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 shutdown_connections 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fc13807ad20 0x7fc13807d1e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14c104520 0x7fc14c199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc14c103b60 0x7fc14c1988f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 --2- 172.21.15.189:0/2105633564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14c102a00 0x7fc14c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 >> 172.21.15.189:0/2105633564 conn(0x7fc14c0fe2f0 msgr2=0x7fc14c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 shutdown_connections 2024-09-13T13:29:00.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:00.662+0000 7fc15353b640 1 -- 172.21.15.189:0/2105633564 wait complete. 2024-09-13T13:29:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[30106]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[30106]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-13T13:29:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[30106]: Cluster is now healthy 2024-09-13T13:29:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/2105633564' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-13T13:29:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-13T13:29:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[40255]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[40255]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-13T13:29:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[40255]: Cluster is now healthy 2024-09-13T13:29:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/2105633564' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-13T13:29:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-13T13:29:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:00 smithi189 ceph-mon[32912]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:00 smithi189 ceph-mon[32912]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-13T13:29:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:00 smithi189 ceph-mon[32912]: Cluster is now healthy 2024-09-13T13:29:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/2105633564' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-13T13:29:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-13T13:29:01.093 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:29:00 smithi043 podman[66770]: 2024-09-13 13:29:00.841136378 +0000 UTC m=+0.131810993 container create cd94fd90adfe7fb981cc80f2273257a3e0dc4a38f09e171eaed5a9692436746b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-13T13:29:01.094 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:29:01 smithi043 podman[66770]: 2024-09-13 13:29:01.093429971 +0000 UTC m=+0.384104543 container init cd94fd90adfe7fb981cc80f2273257a3e0dc4a38f09e171eaed5a9692436746b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-13T13:29:01.379 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-13T13:29:01.491 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:29:01 smithi043 podman[66770]: 2024-09-13 13:29:01.098578355 +0000 UTC m=+0.389252927 container start cd94fd90adfe7fb981cc80f2273257a3e0dc4a38f09e171eaed5a9692436746b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T13:29:01.492 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:29:01 smithi043 bash[66770]: cd94fd90adfe7fb981cc80f2273257a3e0dc4a38f09e171eaed5a9692436746b 2024-09-13T13:29:01.492 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:29:01 smithi043 systemd[1]: Started Ceph rgw.foo.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:29:01.621 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:29:01.934 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-13T13:29:01.934 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T13:29:01.934 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.934 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.934 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.934 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.935 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.935 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:01 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:29:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T13:29:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:01.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:01 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:29:02.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: pgmap v189: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-13T13:29:02.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-13T13:29:02.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:29:02.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T13:29:02.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2421931580' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-13T13:29:02.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/2664534906' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-13T13:29:02.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: pgmap v189: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2421931580' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/2664534906' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-13T13:29:02.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:02 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: pgmap v189: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2421931580' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/2664534906' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-13T13:29:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:02 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2421931580' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:29:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[30106]: Checking dashboard <-> RGW credentials 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2421931580' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:29:03.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:03 smithi043 ceph-mon[40255]: Checking dashboard <-> RGW credentials 2024-09-13T13:29:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2421931580' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-13T13:29:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-13T13:29:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T13:29:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:29:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:03 smithi189 ceph-mon[32912]: Checking dashboard <-> RGW credentials 2024-09-13T13:29:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: pgmap v192: 36 pgs: 9 creating+peering, 25 unknown, 2 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: Cluster is now healthy 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-13T13:29:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[30106]: osdmap e56: 8 total, 8 up, 8 in 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: pgmap v192: 36 pgs: 9 creating+peering, 25 unknown, 2 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: Cluster is now healthy 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-13T13:29:04.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:04 smithi043 ceph-mon[40255]: osdmap e56: 8 total, 8 up, 8 in 2024-09-13T13:29:05.000 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: pgmap v192: 36 pgs: 9 creating+peering, 25 unknown, 2 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T13:29:05.000 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T13:29:05.000 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:05.000 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:05.000 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-13T13:29:05.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T13:29:05.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: Cluster is now healthy 2024-09-13T13:29:05.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-13T13:29:05.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-13T13:29:05.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:04 smithi189 ceph-mon[32912]: osdmap e56: 8 total, 8 up, 8 in 2024-09-13T13:29:05.336 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.189 --placement '1;smithi189=iscsi.a' 2024-09-13T13:29:05.587 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:29:06.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 -- 172.21.15.189:0/4114157529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0104530 msgr2=0x7f10d010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 --2- 172.21.15.189:0/4114157529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0104530 0x7f10d010aeb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009cd0 tx=0x7f10c802a8e0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 -- 172.21.15.189:0/4114157529 shutdown_connections 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 --2- 172.21.15.189:0/4114157529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0104530 0x7f10d010aeb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 --2- 172.21.15.189:0/4114157529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d0103b70 0x7f10d0103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 --2- 172.21.15.189:0/4114157529 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10d0102a10 0x7f10d0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 -- 172.21.15.189:0/4114157529 >> 172.21.15.189:0/4114157529 conn(0x7f10d00fe2c0 msgr2=0x7f10d0100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 -- 172.21.15.189:0/4114157529 shutdown_connections 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.141+0000 7f10d48c5640 1 -- 172.21.15.189:0/4114157529 wait complete. 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 Processor -- start 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 -- start start 2024-09-13T13:29:06.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0102a10 0x7f10d0076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10d0103b70 0x7f10d0075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d0104530 0x7f10d0075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10d010dbe0 con 0x7f10d0104530 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f10d010da60 con 0x7f10d0102a10 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10d48c5640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f10d010dd60 con 0x7f10d0103b70 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10ce59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0102a10 0x7f10d0076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10ce59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0102a10 0x7f10d0076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:50830/0 (socket says 172.21.15.189:50830) 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10ce59c640 1 -- 172.21.15.189:0/512702000 learned_addr learned my addr 172.21.15.189:0/512702000 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.142+0000 7f10ced9d640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d0104530 0x7f10d0075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10c5d9b640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10d0103b70 0x7f10d0075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10ce59c640 1 -- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10d0103b70 msgr2=0x7f10d0075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10ce59c640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10d0103b70 0x7f10d0075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.147 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10ce59c640 1 -- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d0104530 msgr2=0x7f10d0075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10ce59c640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d0104530 0x7f10d0075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10ce59c640 1 -- 172.21.15.189:0/512702000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d0076340 con 0x7f10d0102a10 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10c5d9b640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10d0103b70 0x7f10d0075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10ce59c640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0102a10 0x7f10d0076cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b4a0 tx=0x7f10c000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10c7fff640 1 -- 172.21.15.189:0/512702000 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0014070 con 0x7f10d0102a10 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10c7fff640 1 -- 172.21.15.189:0/512702000 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f10c0010a60 con 0x7f10d0102a10 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d01a24f0 con 0x7f10d0102a10 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10c7fff640 1 -- 172.21.15.189:0/512702000 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c00179e0 con 0x7f10d0102a10 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.143+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01a2a30 con 0x7f10d0102a10 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.144+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d0102e90 con 0x7f10d0102a10 2024-09-13T13:29:06.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.145+0000 7f10c7fff640 1 -- 172.21.15.189:0/512702000 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f10c0017070 con 0x7f10d0102a10 2024-09-13T13:29:06.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.145+0000 7f10c7fff640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10bc07ad20 0x7f10bc07d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:06.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.145+0000 7f10c7fff640 1 -- 172.21.15.189:0/512702000 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6755+0+0 (secure 0 0 0) 0x7f10c009d410 con 0x7f10d0102a10 2024-09-13T13:29:06.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.146+0000 7f10c5d9b640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10bc07ad20 0x7f10bc07d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:06.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.146+0000 7f10c5d9b640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10bc07ad20 0x7f10bc07d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10b8006fd0 tx=0x7f10b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:06.150 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.150+0000 7f10c7fff640 1 -- 172.21.15.189:0/512702000 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f10c0064570 con 0x7f10d0102a10 2024-09-13T13:29:06.338 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.337+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7f10d0063590 con 0x7f10bc07ad20 2024-09-13T13:29:06.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.346+0000 7f10c7fff640 1 -- 172.21.15.189:0/512702000 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (secure 0 0 0) 0x7f10d0063590 con 0x7f10bc07ad20 2024-09-13T13:29:06.347 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled iscsi.datapool update... 2024-09-13T13:29:06.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10bc07ad20 msgr2=0x7f10bc07d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10bc07ad20 0x7f10bc07d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10b8006fd0 tx=0x7f10b8008040 comp rx=0 tx=0).stop 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0102a10 msgr2=0x7f10d0076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0102a10 0x7f10d0076cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b4a0 tx=0x7f10c000b960 comp rx=0 tx=0).stop 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 shutdown_connections 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10bc07ad20 0x7f10bc07d1e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d0104530 0x7f10d0075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10d0103b70 0x7f10d0075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 --2- 172.21.15.189:0/512702000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10d0102a10 0x7f10d0076cd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 >> 172.21.15.189:0/512702000 conn(0x7f10d00fe2c0 msgr2=0x7f10d00fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 shutdown_connections 2024-09-13T13:29:06.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:06.348+0000 7f10d48c5640 1 -- 172.21.15.189:0/512702000 wait complete. 2024-09-13T13:29:06.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:06 smithi189 ceph-mon[32912]: pgmap v195: 68 pgs: 28 creating+peering, 13 unknown, 27 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:29:06.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:06 smithi189 ceph-mon[32912]: osdmap e57: 8 total, 8 up, 8 in 2024-09-13T13:29:06.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:06 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:06.978 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-13T13:29:06.978 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:29:06.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-13T13:29:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[30106]: pgmap v195: 68 pgs: 28 creating+peering, 13 unknown, 27 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:29:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[30106]: osdmap e57: 8 total, 8 up, 8 in 2024-09-13T13:29:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[40255]: pgmap v195: 68 pgs: 28 creating+peering, 13 unknown, 27 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:29:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[40255]: osdmap e57: 8 total, 8 up, 8 in 2024-09-13T13:29:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-13T13:29:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:06 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:07.011 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:29:07.011 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-13T13:29:07.037 DEBUG:teuthology.orchestra.run.smithi189:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-13T13:29:07.081 INFO:tasks.cephadm:Adding prometheus.a on smithi189 2024-09-13T13:29:07.081 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi189=a' 2024-09-13T13:29:07.360 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:29:07.736 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:07 smithi189 ceph-mon[32912]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:07.736 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:07 smithi189 ceph-mon[32912]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-13T13:29:07.736 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-13T13:29:07.736 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-13T13:29:07.736 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:07 smithi189 ceph-mon[32912]: osdmap e58: 8 total, 8 up, 8 in 2024-09-13T13:29:07.852 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 -- 172.21.15.189:0/2702255013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 msgr2=0x7fcf88101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2702255013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf88101d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009930 tx=0x7fcf7802a420 comp rx=0 tx=0).stop 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 -- 172.21.15.189:0/2702255013 shutdown_connections 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2702255013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88103440 0x7fcf88109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2702255013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf88102a80 0x7fcf88102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2702255013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf88101d20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 -- 172.21.15.189:0/2702255013 >> 172.21.15.189:0/2702255013 conn(0x7fcf880fd1d0 msgr2=0x7fcf880ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 -- 172.21.15.189:0/2702255013 shutdown_connections 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.819+0000 7fcf8d657640 1 -- 172.21.15.189:0/2702255013 wait complete. 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 Processor -- start 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 -- start start 2024-09-13T13:29:07.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf8806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf88102a80 0x7fcf8819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88103440 0x7fcf881a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810ca10 con 0x7fcf88103440 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810c890 con 0x7fcf88101920 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf8d657640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8810cb90 con 0x7fcf88102a80 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf86ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf8806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf86ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf8806bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:50850/0 (socket says 172.21.15.189:50850) 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.820+0000 7fcf86ffd640 1 -- 172.21.15.189:0/2566147362 learned_addr learned my addr 172.21.15.189:0/2566147362 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf877fe640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88103440 0x7fcf881a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf7e7fc640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf88102a80 0x7fcf8819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf86ffd640 1 -- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf88102a80 msgr2=0x7fcf8819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:07.854 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf86ffd640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf88102a80 0x7fcf8819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf86ffd640 1 -- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88103440 msgr2=0x7fcf881a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf86ffd640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88103440 0x7fcf881a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf86ffd640 1 -- 172.21.15.189:0/2566147362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf881a2180 con 0x7fcf88101920 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf7e7fc640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf88102a80 0x7fcf8819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf86ffd640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf8806bd00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009a50 tx=0x7fcf78004bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf84ff9640 1 -- 172.21.15.189:0/2566147362 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7803f070 con 0x7fcf88101920 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf881a2410 con 0x7fcf88101920 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf84ff9640 1 -- 172.21.15.189:0/2566147362 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fcf78004de0 con 0x7fcf88101920 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf84ff9640 1 -- 172.21.15.189:0/2566147362 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78035030 con 0x7fcf88101920 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.821+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881a28f0 con 0x7fcf88101920 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.822+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf88101d20 con 0x7fcf88101920 2024-09-13T13:29:07.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.823+0000 7fcf84ff9640 1 -- 172.21.15.189:0/2566147362 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7fcf78042070 con 0x7fcf88101920 2024-09-13T13:29:07.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.823+0000 7fcf84ff9640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fcf7407ad20 0x7fcf7407d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:07.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.823+0000 7fcf84ff9640 1 -- 172.21.15.189:0/2566147362 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7121+0+0 (secure 0 0 0) 0x7fcf780bec70 con 0x7fcf88101920 2024-09-13T13:29:07.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.824+0000 7fcf7e7fc640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fcf7407ad20 0x7fcf7407d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:07.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.824+0000 7fcf7e7fc640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fcf7407ad20 0x7fcf7407d1e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf70005fd0 tx=0x7fcf70005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:07.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:07.828+0000 7fcf84ff9640 1 -- 172.21.15.189:0/2566147362 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fcf7808a040 con 0x7fcf88101920 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[40255]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[40255]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[40255]: osdmap e58: 8 total, 8 up, 8 in 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[30106]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[30106]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-13T13:29:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-13T13:29:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-13T13:29:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:07 smithi043 ceph-mon[30106]: osdmap e58: 8 total, 8 up, 8 in 2024-09-13T13:29:08.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.018+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}) -- 0x7fcf8802e8f0 con 0x7fcf7407ad20 2024-09-13T13:29:08.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.025+0000 7fcf84ff9640 1 -- 172.21.15.189:0/2566147362 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (secure 0 0 0) 0x7fcf8802e8f0 con 0x7fcf7407ad20 2024-09-13T13:29:08.026 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled prometheus update... 2024-09-13T13:29:08.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fcf7407ad20 msgr2=0x7fcf7407d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:08.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fcf7407ad20 0x7fcf7407d1e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf70005fd0 tx=0x7fcf70005ea0 comp rx=0 tx=0).stop 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 msgr2=0x7fcf8806bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf8806bd00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009a50 tx=0x7fcf78004bf0 comp rx=0 tx=0).stop 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 shutdown_connections 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fcf7407ad20 0x7fcf7407d1e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88103440 0x7fcf881a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf88102a80 0x7fcf8819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 --2- 172.21.15.189:0/2566147362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf88101920 0x7fcf8806bd00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 >> 172.21.15.189:0/2566147362 conn(0x7fcf880fd1d0 msgr2=0x7fcf880feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 shutdown_connections 2024-09-13T13:29:08.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:08.027+0000 7fcf8d657640 1 -- 172.21.15.189:0/2566147362 wait complete. 2024-09-13T13:29:08.659 DEBUG:teuthology.orchestra.run.smithi189:prometheus.a> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-13T13:29:08.662 INFO:tasks.cephadm:Adding node-exporter.a on smithi043 2024-09-13T13:29:08.662 INFO:tasks.cephadm:Adding node-exporter.b on smithi189 2024-09-13T13:29:08.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi043=a;smithi189=b' 2024-09-13T13:29:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:08 smithi189 ceph-mon[32912]: pgmap v198: 100 pgs: 28 creating+peering, 45 unknown, 27 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:29:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:08 smithi189 ceph-mon[32912]: osdmap e59: 8 total, 8 up, 8 in 2024-09-13T13:29:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:08 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:08.950 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:29:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[30106]: pgmap v198: 100 pgs: 28 creating+peering, 45 unknown, 27 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:29:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[30106]: osdmap e59: 8 total, 8 up, 8 in 2024-09-13T13:29:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[40255]: pgmap v198: 100 pgs: 28 creating+peering, 45 unknown, 27 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:29:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[40255]: osdmap e59: 8 total, 8 up, 8 in 2024-09-13T13:29:08.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-13T13:29:08.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:08 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 -- 172.21.15.189:0/1571788956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c103430 msgr2=0x7fe61c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 --2- 172.21.15.189:0/1571788956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c103430 0x7fe61c109db0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe614009cd0 tx=0x7fe61402a8e0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 -- 172.21.15.189:0/1571788956 shutdown_connections 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 --2- 172.21.15.189:0/1571788956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c103430 0x7fe61c109db0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 --2- 172.21.15.189:0/1571788956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe61c102a70 0x7fe61c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 --2- 172.21.15.189:0/1571788956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe61c101910 0x7fe61c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 -- 172.21.15.189:0/1571788956 >> 172.21.15.189:0/1571788956 conn(0x7fe61c0fd200 msgr2=0x7fe61c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 -- 172.21.15.189:0/1571788956 shutdown_connections 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 -- 172.21.15.189:0/1571788956 wait complete. 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.391+0000 7fe6215ac640 1 Processor -- start 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe6215ac640 1 -- start start 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe6215ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c101910 0x7fe61c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe6215ac640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe61c102a70 0x7fe61c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe6215ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe61c103430 0x7fe61c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe6215ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c10cae0 con 0x7fe61c103430 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe6215ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c10c960 con 0x7fe61c101910 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe6215ac640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c10cc60 con 0x7fe61c102a70 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c101910 0x7fe61c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:09.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c101910 0x7fe61c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:50876/0 (socket says 172.21.15.189:50876) 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 -- 172.21.15.189:0/4219637193 learned_addr learned my addr 172.21.15.189:0/4219637193 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61a7fc640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe61c102a70 0x7fe61c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61b7fe640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe61c103430 0x7fe61c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 -- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe61c102a70 msgr2=0x7fe61c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe61c102a70 0x7fe61c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 -- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe61c103430 msgr2=0x7fe61c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe61c103430 0x7fe61c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 -- 172.21.15.189:0/4219637193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c19a000 con 0x7fe61c101910 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.392+0000 7fe61affd640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c101910 0x7fe61c1984d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b4a0 tx=0x7fe60c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.393+0000 7fe603fff640 1 -- 172.21.15.189:0/4219637193 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c015020 con 0x7fe61c101910 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.393+0000 7fe603fff640 1 -- 172.21.15.189:0/4219637193 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fe60c011c80 con 0x7fe61c101910 2024-09-13T13:29:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.393+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c19a2f0 con 0x7fe61c101910 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.393+0000 7fe603fff640 1 -- 172.21.15.189:0/4219637193 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c010bb0 con 0x7fe61c101910 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.393+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c1a09d0 con 0x7fe61c101910 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.394+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c101d90 con 0x7fe61c101910 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.394+0000 7fe603fff640 1 -- 172.21.15.189:0/4219637193 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7fe60c018020 con 0x7fe61c101910 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.395+0000 7fe603fff640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fe60407ad20 0x7fe60407d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.395+0000 7fe603fff640 1 -- 172.21.15.189:0/4219637193 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7132+0+0 (secure 0 0 0) 0x7fe60c09d950 con 0x7fe61c101910 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.395+0000 7fe61a7fc640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fe60407ad20 0x7fe60407d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.395+0000 7fe61a7fc640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fe60407ad20 0x7fe60407d1e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe610009990 tx=0x7fe610009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:09.415 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.399+0000 7fe603fff640 1 -- 172.21.15.189:0/4219637193 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fe60c01f020 con 0x7fe61c101910 2024-09-13T13:29:09.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.590+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi189=b", "target": ["mon-mgr", ""]}) -- 0x7fe61c063590 con 0x7fe60407ad20 2024-09-13T13:29:09.599 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.598+0000 7fe603fff640 1 -- 172.21.15.189:0/4219637193 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7fe61c063590 con 0x7fe60407ad20 2024-09-13T13:29:09.599 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled node-exporter update... 2024-09-13T13:29:09.601 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fe60407ad20 msgr2=0x7fe60407d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:09.601 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fe60407ad20 0x7fe60407d1e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe610009990 tx=0x7fe610009110 comp rx=0 tx=0).stop 2024-09-13T13:29:09.601 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c101910 msgr2=0x7fe61c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:09.601 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c101910 0x7fe61c1984d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b4a0 tx=0x7fe60c00b8a0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.601 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 shutdown_connections 2024-09-13T13:29:09.601 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fe60407ad20 0x7fe60407d1e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.602 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe61c103430 0x7fe61c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.602 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe61c102a70 0x7fe61c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.602 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 --2- 172.21.15.189:0/4219637193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61c101910 0x7fe61c1984d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:09.602 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 >> 172.21.15.189:0/4219637193 conn(0x7fe61c0fd200 msgr2=0x7fe61c06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:09.602 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 shutdown_connections 2024-09-13T13:29:09.602 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:09.600+0000 7fe6215ac640 1 -- 172.21.15.189:0/4219637193 wait complete. 2024-09-13T13:29:09.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:09.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-13T13:29:09.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-13T13:29:09.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: osdmap e60: 8 total, 8 up, 8 in 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-13T13:29:09.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:09 smithi189 ceph-mon[32912]: osdmap e61: 8 total, 8 up, 8 in 2024-09-13T13:29:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: osdmap e60: 8 total, 8 up, 8 in 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-13T13:29:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[40255]: osdmap e61: 8 total, 8 up, 8 in 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: osdmap e60: 8 total, 8 up, 8 in 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3531949875' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3954660948' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-13T13:29:09.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:09 smithi043 ceph-mon[30106]: osdmap e61: 8 total, 8 up, 8 in 2024-09-13T13:29:10.401 DEBUG:teuthology.orchestra.run.smithi043:node-exporter.a> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-13T13:29:10.404 DEBUG:teuthology.orchestra.run.smithi189:node-exporter.b> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-13T13:29:10.406 INFO:tasks.cephadm:Adding alertmanager.a on smithi043 2024-09-13T13:29:10.406 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi043=a' 2024-09-13T13:29:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: pgmap v201: 132 pgs: 20 creating+peering, 33 unknown, 79 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:29:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: Saving service node-exporter spec with placement smithi043=a;smithi189=b;count:2 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: pgmap v201: 132 pgs: 20 creating+peering, 33 unknown, 79 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: Saving service node-exporter spec with placement smithi043=a;smithi189=b;count:2 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:10 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: pgmap v201: 132 pgs: 20 creating+peering, 33 unknown, 79 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: Saving service node-exporter spec with placement smithi043=a;smithi189=b;count:2 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.994 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.995 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:10.995 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-13T13:29:10.995 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-13T13:29:10.995 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:10 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:11.210 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:29:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:11 smithi043 ceph-mon[30106]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-13T13:29:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:11 smithi043 ceph-mon[40255]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-13T13:29:12.009 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:11 smithi189 ceph-mon[32912]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.094+0000 7f8a14376640 1 -- 172.21.15.189:0/4087029945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c101910 msgr2=0x7f8a0c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.094+0000 7f8a14376640 1 --2- 172.21.15.189:0/4087029945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c101910 0x7f8a0c101d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009930 tx=0x7f8a0402a420 comp rx=0 tx=0).stop 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- 172.21.15.189:0/4087029945 shutdown_connections 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 --2- 172.21.15.189:0/4087029945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a0c103430 0x7f8a0c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 --2- 172.21.15.189:0/4087029945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a0c102a70 0x7f8a0c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 --2- 172.21.15.189:0/4087029945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c101910 0x7f8a0c101d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- 172.21.15.189:0/4087029945 >> 172.21.15.189:0/4087029945 conn(0x7f8a0c0fd200 msgr2=0x7f8a0c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- 172.21.15.189:0/4087029945 shutdown_connections 2024-09-13T13:29:12.143 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- 172.21.15.189:0/4087029945 wait complete. 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 Processor -- start 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- start start 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a0c101910 0x7f8a0c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c102a70 0x7f8a0c197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a0c103430 0x7f8a0c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c10cae0 con 0x7f8a0c101910 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c10c960 con 0x7f8a0c102a70 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.095+0000 7f8a14376640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c10cc60 con 0x7f8a0c103430 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c102a70 0x7f8a0c197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c102a70 0x7f8a0c197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:56584/0 (socket says 172.21.15.189:56584) 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 -- 172.21.15.189:0/2460281752 learned_addr learned my addr 172.21.15.189:0/2460281752 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a12112640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a0c101910 0x7f8a0c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:12.144 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a12913640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a0c103430 0x7f8a0c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 -- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a0c103430 msgr2=0x7f8a0c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a0c103430 0x7f8a0c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 -- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a0c101910 msgr2=0x7f8a0c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a0c101910 0x7f8a0c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 -- 172.21.15.189:0/2460281752 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c198f50 con 0x7f8a0c102a70 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a12112640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a0c101910 0x7f8a0c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a12913640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a0c103430 0x7f8a0c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a11911640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c102a70 0x7f8a0c197a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000d990 tx=0x7f8a0000de50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a0b7fe640 1 -- 172.21.15.189:0/2460281752 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0001b070 con 0x7f8a0c102a70 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a0b7fe640 1 -- 172.21.15.189:0/2460281752 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f8a0000edf0 con 0x7f8a0c102a70 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a0b7fe640 1 -- 172.21.15.189:0/2460281752 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000145a0 con 0x7f8a0c102a70 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c199240 con 0x7f8a0c102a70 2024-09-13T13:29:12.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.096+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c06b650 con 0x7f8a0c102a70 2024-09-13T13:29:12.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.097+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c101d90 con 0x7f8a0c102a70 2024-09-13T13:29:12.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.098+0000 7f8a0b7fe640 1 -- 172.21.15.189:0/2460281752 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f8a0001e070 con 0x7f8a0c102a70 2024-09-13T13:29:12.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.099+0000 7f8a0b7fe640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f89f407ad20 0x7f89f407d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:12.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.099+0000 7f8a0b7fe640 1 -- 172.21.15.189:0/2460281752 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f8a0009f820 con 0x7f8a0c102a70 2024-09-13T13:29:12.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.099+0000 7f8a12112640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f89f407ad20 0x7f89f407d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:12.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.100+0000 7f8a12112640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f89f407ad20 0x7f89f407d1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a040315c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:12.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.103+0000 7f8a0b7fe640 1 -- 172.21.15.189:0/2460281752 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f8a0006abe0 con 0x7f8a0c102a70 2024-09-13T13:29:12.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.311+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}) -- 0x7f8a0c06b910 con 0x7f89f407ad20 2024-09-13T13:29:12.320 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.319+0000 7f8a0b7fe640 1 -- 172.21.15.189:0/2460281752 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (secure 0 0 0) 0x7f8a0c06b910 con 0x7f89f407ad20 2024-09-13T13:29:12.320 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled alertmanager update... 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f89f407ad20 msgr2=0x7f89f407d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f89f407ad20 0x7f89f407d1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a040315c0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c102a70 msgr2=0x7f8a0c197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c102a70 0x7f8a0c197a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000d990 tx=0x7f8a0000de50 comp rx=0 tx=0).stop 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 shutdown_connections 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f89f407ad20 0x7f89f407d1e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a0c103430 0x7f8a0c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a0c102a70 0x7f8a0c197a70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.324 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 --2- 172.21.15.189:0/2460281752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a0c101910 0x7f8a0c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:12.324 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 >> 172.21.15.189:0/2460281752 conn(0x7f8a0c0fd200 msgr2=0x7f8a0c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:12.324 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 shutdown_connections 2024-09-13T13:29:12.324 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:12.322+0000 7f8a14376640 1 -- 172.21.15.189:0/2460281752 wait complete. 2024-09-13T13:29:12.638 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:12 smithi189 systemd[1]: Starting Ceph iscsi.iscsi.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:29:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:12 smithi189 ceph-mon[32912]: pgmap v203: 132 pgs: 6 creating+peering, 12 unknown, 114 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-13T13:29:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:12 smithi189 ceph-mon[32912]: osdmap e62: 8 total, 8 up, 8 in 2024-09-13T13:29:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:12 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:12.983 DEBUG:teuthology.orchestra.run.smithi043:alertmanager.a> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-13T13:29:12.986 INFO:tasks.cephadm:Adding grafana.a on smithi189 2024-09-13T13:29:12.986 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi189=a' 2024-09-13T13:29:13.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:12 smithi043 ceph-mon[30106]: pgmap v203: 132 pgs: 6 creating+peering, 12 unknown, 114 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-13T13:29:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:12 smithi043 ceph-mon[30106]: osdmap e62: 8 total, 8 up, 8 in 2024-09-13T13:29:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:12 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:12 smithi043 ceph-mon[40255]: pgmap v203: 132 pgs: 6 creating+peering, 12 unknown, 114 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-13T13:29:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:12 smithi043 ceph-mon[40255]: osdmap e62: 8 total, 8 up, 8 in 2024-09-13T13:29:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:12 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:13.316 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 podman[59821]: 2024-09-13 13:29:13.047688709 +0000 UTC m=+0.016521264 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:29:13.316 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 podman[59821]: 2024-09-13 13:29:13.198332756 +0000 UTC m=+0.167165298 container create 0c06c8b206d63384487c3de7cd29ba89561bdcf61a5e194893fb6fe43ae76d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T13:29:13.548 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:29:13.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 podman[59821]: 2024-09-13 13:29:13.408900902 +0000 UTC m=+0.377733449 container init 0c06c8b206d63384487c3de7cd29ba89561bdcf61a5e194893fb6fe43ae76d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True) 2024-09-13T13:29:13.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 podman[59821]: 2024-09-13 13:29:13.413545613 +0000 UTC m=+0.382378164 container start 0c06c8b206d63384487c3de7cd29ba89561bdcf61a5e194893fb6fe43ae76d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:29:13.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 bash[59821]: 0c06c8b206d63384487c3de7cd29ba89561bdcf61a5e194893fb6fe43ae76d62 2024-09-13T13:29:13.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 systemd[1]: Started Ceph iscsi.iscsi.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:29:13.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug Started the configuration object watcher 2024-09-13T13:29:13.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug Checking for config object changes every 1s 2024-09-13T13:29:13.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug Processing osd blocklist entries for this node 2024-09-13T13:29:13.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-mon[32912]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:13.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-mon[32912]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-13T13:29:13.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:13.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:13.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:13.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:13 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[30106]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[30106]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[40255]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[40255]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:13 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug Reading the configuration object to update local LIO configuration 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug Configuration does not have an entry for this host(smithi189.front.sepia.ceph.com) - nothing to define to LIO 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: * Environment: production 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: Use a production WSGI server instead. 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: * Debug mode: off 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug * Running on all addresses. 2024-09-13T13:29:14.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-13T13:29:14.270 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: * Running on all addresses. 2024-09-13T13:29:14.270 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-13T13:29:14.270 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug * Running on http://[fe80::ec4:7aff:fe88:6df1]:5000/ (Press CTRL+C to quit) 2024-09-13T13:29:14.270 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: * Running on http://[fe80::ec4:7aff:fe88:6df1]:5000/ (Press CTRL+C to quit) 2024-09-13T13:29:14.298 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 -- 172.21.15.189:0/649069468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c103400 msgr2=0x7f392c109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 --2- 172.21.15.189:0/649069468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c103400 0x7f392c109d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3924009930 tx=0x7f392402a420 comp rx=0 tx=0).stop 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 -- 172.21.15.189:0/649069468 shutdown_connections 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 --2- 172.21.15.189:0/649069468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c103400 0x7f392c109d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 --2- 172.21.15.189:0/649069468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392c102a40 0x7f392c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 --2- 172.21.15.189:0/649069468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f392c1018e0 0x7f392c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 -- 172.21.15.189:0/649069468 >> 172.21.15.189:0/649069468 conn(0x7f392c0fd1b0 msgr2=0x7f392c0ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 -- 172.21.15.189:0/649069468 shutdown_connections 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 -- 172.21.15.189:0/649069468 wait complete. 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 Processor -- start 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.267+0000 7f393329c640 1 -- start start 2024-09-13T13:29:14.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f393329c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c1018e0 0x7f392c10c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f393329c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f392c102a40 0x7f392c10a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f393329c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392c103400 0x7f392c10af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f393329c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f392c10db50 con 0x7f392c103400 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f393329c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f392c10d9d0 con 0x7f392c1018e0 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f393329c640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f392c10dcd0 con 0x7f392c102a40 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c1018e0 0x7f392c10c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3930837640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f392c102a40 0x7f392c10a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392c103400 0x7f392c10af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c1018e0 0x7f392c10c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:56656/0 (socket says 172.21.15.189:56656) 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3930837640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f392c102a40 0x7f392c10a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.189:55130/0 (socket says 172.21.15.189:55130) 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 -- 172.21.15.189:0/780476759 learned_addr learned my addr 172.21.15.189:0/780476759 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:29:14.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 -- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f392c102a40 msgr2=0x7f392c10a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f392c102a40 0x7f392c10a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 -- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392c103400 msgr2=0x7f392c10af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392c103400 0x7f392c10af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 -- 172.21.15.189:0/780476759 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c10b7d0 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f3931038640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c1018e0 0x7f392c10c160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f391c00ba50 tx=0x7f391c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.268+0000 7f392a7fc640 1 -- 172.21.15.189:0/780476759 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c019070 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.269+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392c1a2420 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.269+0000 7f392a7fc640 1 -- 172.21.15.189:0/780476759 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f391c00cbe0 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.269+0000 7f392a7fc640 1 -- 172.21.15.189:0/780476759 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c007390 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.269+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392c1a2960 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.270+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392c101ce0 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.270+0000 7f392a7fc640 1 -- 172.21.15.189:0/780476759 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 103079+0+0 (secure 0 0 0) 0x7f391c01c070 con 0x7f392c1018e0 2024-09-13T13:29:14.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.271+0000 7f392a7fc640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f391407ad20 0x7f391407d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:14.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.271+0000 7f392a7fc640 1 -- 172.21.15.189:0/780476759 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f391c09d760 con 0x7f392c1018e0 2024-09-13T13:29:14.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.271+0000 7f3930837640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f391407ad20 0x7f391407d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:14.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.271+0000 7f3930837640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f391407ad20 0x7f391407d1e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39200058c0 tx=0x7f392000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:14.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.275+0000 7f392a7fc640 1 -- 172.21.15.189:0/780476759 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f391c068b20 con 0x7f392c1018e0 2024-09-13T13:29:14.470 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.469+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 --> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}) -- 0x7f392c063590 con 0x7f391407ad20 2024-09-13T13:29:14.478 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.477+0000 7f392a7fc640 1 -- 172.21.15.189:0/780476759 <== mgr.14152 v2:172.21.15.43:6800/1020071518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f392c063590 con 0x7f391407ad20 2024-09-13T13:29:14.478 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled grafana update... 2024-09-13T13:29:14.480 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f391407ad20 msgr2=0x7f391407d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:14.480 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f391407ad20 0x7f391407d1e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39200058c0 tx=0x7f392000a2d0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.480 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c1018e0 msgr2=0x7f392c10c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c1018e0 0x7f392c10c160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f391c00ba50 tx=0x7f391c00bf10 comp rx=0 tx=0).stop 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 shutdown_connections 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f391407ad20 0x7f391407d1e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392c103400 0x7f392c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f392c102a40 0x7f392c10a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 --2- 172.21.15.189:0/780476759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f392c1018e0 0x7f392c10c160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 >> 172.21.15.189:0/780476759 conn(0x7f392c0fd1b0 msgr2=0x7f392c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 shutdown_connections 2024-09-13T13:29:14.481 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:14.479+0000 7f393329c640 1 -- 172.21.15.189:0/780476759 wait complete. 2024-09-13T13:29:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-mon[32912]: pgmap v205: 132 pgs: 6 creating+peering, 126 active+clean; 580 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 3.1 KiB/s wr, 86 op/s 2024-09-13T13:29:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-mon[32912]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-13T13:29:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-mon[32912]: Deploying daemon prometheus.a on smithi189 2024-09-13T13:29:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/3923407660' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-13T13:29:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:14 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[40255]: pgmap v205: 132 pgs: 6 creating+peering, 126 active+clean; 580 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 3.1 KiB/s wr, 86 op/s 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[40255]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[40255]: Deploying daemon prometheus.a on smithi189 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/3923407660' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[30106]: pgmap v205: 132 pgs: 6 creating+peering, 126 active+clean; 580 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 3.1 KiB/s wr, 86 op/s 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[30106]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[30106]: Deploying daemon prometheus.a on smithi189 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/3923407660' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-13T13:29:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:14 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:15.286 DEBUG:teuthology.orchestra.run.smithi189:grafana.a> sudo journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@grafana.a.service 2024-09-13T13:29:15.288 INFO:tasks.cephadm:Setting up client nodes... 2024-09-13T13:29:15.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T13:29:15.558 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:15.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:15 smithi043 ceph-mon[30106]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:15.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:15 smithi043 ceph-mon[30106]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-13T13:29:15.950 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:15 smithi043 ceph-mon[40255]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:15.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:15 smithi043 ceph-mon[40255]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-13T13:29:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 -- 172.21.15.43:0/4024225181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 msgr2=0x7f79001079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 --2- 172.21.15.43:0/4024225181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 0x7f79001079c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009860 tx=0x7f78fc02a340 comp rx=0 tx=0).stop 2024-09-13T13:29:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 -- 172.21.15.43:0/4024225181 shutdown_connections 2024-09-13T13:29:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 --2- 172.21.15.43:0/4024225181 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f790010a910 0x7f790010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 --2- 172.21.15.43:0/4024225181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7900107f00 0x7f790010a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 --2- 172.21.15.43:0/4024225181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 0x7f79001079c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 -- 172.21.15.43:0/4024225181 >> 172.21.15.43:0/4024225181 conn(0x7f79000fd1b0 msgr2=0x7f79000ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 -- 172.21.15.43:0/4024225181 shutdown_connections 2024-09-13T13:29:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.044+0000 7f7907ea9640 1 -- 172.21.15.43:0/4024225181 wait complete. 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 Processor -- start 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 -- start start 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 0x7f790019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7900107f00 0x7f790019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f790010a910 0x7f790019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f790010fce0 con 0x7f790006b5a0 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f790010fb60 con 0x7f790010a910 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7907ea9640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f790010fe60 con 0x7f7900107f00 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905c45640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 0x7f790019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905444640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7900107f00 0x7f790019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905c45640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 0x7f790019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59134/0 (socket says 172.21.15.43:59134) 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905c45640 1 -- 172.21.15.43:0/827180110 learned_addr learned my addr 172.21.15.43:0/827180110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7906446640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f790010a910 0x7f790019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905444640 1 -- 172.21.15.43:0/827180110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f790010a910 msgr2=0x7f790019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905444640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f790010a910 0x7f790019b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905444640 1 -- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 msgr2=0x7f790019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905444640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 0x7f790019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7905444640 1 -- 172.21.15.43:0/827180110 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790019c0a0 con 0x7f7900107f00 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.045+0000 7f7906446640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f790010a910 0x7f790019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.046+0000 7f7905444640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7900107f00 0x7f790019abc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78f000e9c0 tx=0x7f78f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.046+0000 7f78f6ffd640 1 -- 172.21.15.43:0/827180110 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f000ccc0 con 0x7f7900107f00 2024-09-13T13:29:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.046+0000 7f78f6ffd640 1 -- 172.21.15.43:0/827180110 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f78f0010880 con 0x7f7900107f00 2024-09-13T13:29:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.046+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790019c390 con 0x7f7900107f00 2024-09-13T13:29:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.046+0000 7f78f6ffd640 1 -- 172.21.15.43:0/827180110 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0018680 con 0x7f7900107f00 2024-09-13T13:29:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.046+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001a2a70 con 0x7f7900107f00 2024-09-13T13:29:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.047+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79001079c0 con 0x7f7900107f00 2024-09-13T13:29:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.047+0000 7f78f6ffd640 1 -- 172.21.15.43:0/827180110 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f78f0010430 con 0x7f7900107f00 2024-09-13T13:29:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.048+0000 7f78f6ffd640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f78ec07adb0 0x7f78ec07d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.048+0000 7f7905c45640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f78ec07adb0 0x7f78ec07d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.048+0000 7f78f6ffd640 1 -- 172.21.15.43:0/827180110 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f78f009e690 con 0x7f7900107f00 2024-09-13T13:29:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.049+0000 7f7905c45640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f78ec07adb0 0x7f78ec07d270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0095f0 tx=0x7f78fc0056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.053+0000 7f78f6ffd640 1 -- 172.21.15.43:0/827180110 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f78f00698d0 con 0x7f7900107f00 2024-09-13T13:29:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:15 smithi189 ceph-mon[32912]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T13:29:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:15 smithi189 ceph-mon[32912]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-13T13:29:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.311+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f79001a2ca0 con 0x7f7900107f00 2024-09-13T13:29:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.316+0000 7f78f6ffd640 1 -- 172.21.15.43:0/827180110 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) ==== 170+0+59 (secure 0 0 0) 0x7f78f006e780 con 0x7f7900107f00 2024-09-13T13:29:16.317 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-13T13:29:16.317 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAsPuRmHMqoEhAAmOlcTTIYcw+OkhfV2htr/g== 2024-09-13T13:29:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f78ec07adb0 msgr2=0x7f78ec07d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f78ec07adb0 0x7f78ec07d270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0095f0 tx=0x7f78fc0056d0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7900107f00 msgr2=0x7f790019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7900107f00 0x7f790019abc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78f000e9c0 tx=0x7f78f000ee80 comp rx=0 tx=0).stop 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 shutdown_connections 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f78ec07adb0 0x7f78ec07d270 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f790010a910 0x7f790019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7900107f00 0x7f790019abc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 --2- 172.21.15.43:0/827180110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f790006b5a0 0x7f790019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 >> 172.21.15.43:0/827180110 conn(0x7f79000fd1b0 msgr2=0x7f7900102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 shutdown_connections 2024-09-13T13:29:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:16.318+0000 7f7907ea9640 1 -- 172.21.15.43:0/827180110 wait complete. 2024-09-13T13:29:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:16 smithi189 ceph-mon[32912]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s rd, 5.5 KiB/s wr, 172 op/s 2024-09-13T13:29:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:16 smithi189 ceph-mon[32912]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-13T13:29:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/827180110' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T13:29:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[40255]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s rd, 5.5 KiB/s wr, 172 op/s 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[40255]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/827180110' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[30106]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s rd, 5.5 KiB/s wr, 172 op/s 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[30106]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/827180110' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T13:29:17.050 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:29:17.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-13T13:29:17.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T13:29:17.089 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T13:29:17.463 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 -- 172.21.15.189:0/3340223668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db81018e0 msgr2=0x7f1db8101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 --2- 172.21.15.189:0/3340223668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db81018e0 0x7f1db8101ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009930 tx=0x7f1db402a420 comp rx=0 tx=0).stop 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 -- 172.21.15.189:0/3340223668 shutdown_connections 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 --2- 172.21.15.189:0/3340223668 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1db8103400 0x7f1db8109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 --2- 172.21.15.189:0/3340223668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1db8102a40 0x7f1db8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 --2- 172.21.15.189:0/3340223668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db81018e0 0x7f1db8101ce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 -- 172.21.15.189:0/3340223668 >> 172.21.15.189:0/3340223668 conn(0x7f1db80fd1b0 msgr2=0x7f1db80ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 -- 172.21.15.189:0/3340223668 shutdown_connections 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.052+0000 7f1dc0024640 1 -- 172.21.15.189:0/3340223668 wait complete. 2024-09-13T13:29:18.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 Processor -- start 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 -- start start 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1db81018e0 0x7f1db8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db8102a40 0x7f1db8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1db8103400 0x7f1db81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1db810db50 con 0x7f1db8103400 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1db810d9d0 con 0x7f1db8102a40 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dc0024640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f1db810dcd0 con 0x7f1db81018e0 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db8102a40 0x7f1db8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db8102a40 0x7f1db8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:56664/0 (socket says 172.21.15.189:56664) 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 -- 172.21.15.189:0/1135022128 learned_addr learned my addr 172.21.15.189:0/1135022128 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbe5c1640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1db8103400 0x7f1db81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:18.112 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbddc0640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1db81018e0 0x7f1db8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 -- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1db81018e0 msgr2=0x7f1db8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1db81018e0 0x7f1db8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 -- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1db8103400 msgr2=0x7f1db81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1db8103400 0x7f1db81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 -- 172.21.15.189:0/1135022128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db8199e60 con 0x7f1db8102a40 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbe5c1640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1db8103400 0x7f1db81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.053+0000 7f1dbd5bf640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db8102a40 0x7f1db8198980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00d950 tx=0x7f1dac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.054+0000 7f1daaffd640 1 -- 172.21.15.189:0/1135022128 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac017070 con 0x7f1db8102a40 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.054+0000 7f1daaffd640 1 -- 172.21.15.189:0/1135022128 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f1dac009c30 con 0x7f1db8102a40 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.054+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db819a150 con 0x7f1db8102a40 2024-09-13T13:29:18.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.054+0000 7f1daaffd640 1 -- 172.21.15.189:0/1135022128 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00f3c0 con 0x7f1db8102a40 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.054+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db81a0830 con 0x7f1db8102a40 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.055+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db8101ce0 con 0x7f1db8102a40 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.055+0000 7f1daaffd640 1 -- 172.21.15.189:0/1135022128 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f1dac010460 con 0x7f1db8102a40 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.056+0000 7f1daaffd640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f1da407adb0 0x7f1da407d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.056+0000 7f1daaffd640 1 -- 172.21.15.189:0/1135022128 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f1dac09dfc0 con 0x7f1db8102a40 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.056+0000 7f1dbddc0640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f1da407adb0 0x7f1da407d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.057+0000 7f1dbddc0640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f1da407adb0 0x7f1da407d270 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:18.114 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.060+0000 7f1daaffd640 1 -- 172.21.15.189:0/1135022128 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f1dac064ea0 con 0x7f1db8102a40 2024-09-13T13:29:18.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.307+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1db8108480 con 0x7f1db8102a40 2024-09-13T13:29:18.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.312+0000 7f1daaffd640 1 -- 172.21.15.189:0/1135022128 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7f1dac069280 con 0x7f1db8102a40 2024-09-13T13:29:18.313 INFO:teuthology.orchestra.run.smithi189.stdout:[client.1] 2024-09-13T13:29:18.313 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAuPuRmW790EhAAFAi7No7kRrhtthDXzc2DTQ== 2024-09-13T13:29:18.315 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f1da407adb0 msgr2=0x7f1da407d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f1da407adb0 0x7f1da407d270 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4009560 comp rx=0 tx=0).stop 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db8102a40 msgr2=0x7f1db8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db8102a40 0x7f1db8198980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00d950 tx=0x7f1dac00de10 comp rx=0 tx=0).stop 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 shutdown_connections 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f1da407adb0 0x7f1da407d270 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1db8103400 0x7f1db81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db8102a40 0x7f1db8198980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 --2- 172.21.15.189:0/1135022128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1db81018e0 0x7f1db8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.314+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 >> 172.21.15.189:0/1135022128 conn(0x7f1db80fd1b0 msgr2=0x7f1db80fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.315+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 shutdown_connections 2024-09-13T13:29:18.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-13T13:29:18.315+0000 7f1dc0024640 1 -- 172.21.15.189:0/1135022128 wait complete. 2024-09-13T13:29:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:18 smithi189 ceph-mon[32912]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-09-13T13:29:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:18 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:18.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.189:0/1135022128' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:18.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:18.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[40255]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.189:0/1135022128' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[30106]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.189:0/1135022128' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:18.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T13:29:18.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T13:29:19.348 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-13T13:29:19.348 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-13T13:29:19.348 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T13:29:19.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph config log 1 --format=json 2024-09-13T13:29:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 -- 172.21.15.43:0/293210132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 msgr2=0x7f9c7c10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 --2- 172.21.15.43:0/293210132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c10f910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009be0 tx=0x7f9c7402a6c0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 -- 172.21.15.43:0/293210132 shutdown_connections 2024-09-13T13:29:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 --2- 172.21.15.43:0/293210132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c10f910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 --2- 172.21.15.43:0/293210132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c7c108790 0x7f9c7c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 --2- 172.21.15.43:0/293210132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c102d80 0x7f9c7c103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 -- 172.21.15.43:0/293210132 >> 172.21.15.43:0/293210132 conn(0x7f9c7c0fe7d0 msgr2=0x7f9c7c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 -- 172.21.15.43:0/293210132 shutdown_connections 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.151+0000 7f9c82642640 1 -- 172.21.15.43:0/293210132 wait complete. 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 Processor -- start 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 -- start start 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c7c102d80 0x7f9c7c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c108790 0x7f9c7c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c19e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c112260 con 0x7f9c7c108790 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c1120e0 con 0x7f9c7c102d80 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c82642640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c1123e0 con 0x7f9c7c109110 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c108790 0x7f9c7c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c80bdf640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c19e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c108790 0x7f9c7c198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44520/0 (socket says 172.21.15.43:44520) 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c7b7fe640 1 -- 172.21.15.43:0/3135885175 learned_addr learned my addr 172.21.15.43:0/3135885175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.152+0000 7f9c80bdf640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c19e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:33740/0 (socket says 172.21.15.43:33740) 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c7bfff640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c7c102d80 0x7f9c7c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c80bdf640 1 -- 172.21.15.43:0/3135885175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c7c102d80 msgr2=0x7f9c7c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c80bdf640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c7c102d80 0x7f9c7c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c80bdf640 1 -- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c108790 msgr2=0x7f9c7c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c80bdf640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c108790 0x7f9c7c198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c80bdf640 1 -- 172.21.15.43:0/3135885175 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c19e820 con 0x7f9c7c109110 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c7b7fe640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c108790 0x7f9c7c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c80bdf640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c19e050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009b30 tx=0x7f9c74030a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c797fa640 1 -- 172.21.15.43:0/3135885175 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74041020 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c19eab0 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c797fa640 1 -- 172.21.15.43:0/3135885175 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9c74003970 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c797fa640 1 -- 172.21.15.43:0/3135885175 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74036040 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.153+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c1a51d0 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.155+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c103160 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.155+0000 7f9c797fa640 1 -- 172.21.15.43:0/3135885175 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f9c74044020 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.156+0000 7f9c797fa640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9c6407adb0 0x7f9c6407d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.156+0000 7f9c7bfff640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9c6407adb0 0x7f9c6407d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.156+0000 7f9c797fa640 1 -- 172.21.15.43:0/3135885175 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f9c740bedc0 con 0x7f9c7c109110 2024-09-13T13:29:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.156+0000 7f9c7bfff640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9c6407adb0 0x7f9c6407d270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009530 tx=0x7f9c6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:20.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.160+0000 7f9c797fa640 1 -- 172.21.15.43:0/3135885175 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f9c7404b020 con 0x7f9c7c109110 2024-09-13T13:29:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.347+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f9c7c10aef0 con 0x7f9c7c109110 2024-09-13T13:29:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.347+0000 7f9c797fa640 1 -- 172.21.15.43:0/3135885175 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v23) ==== 86+0+187 (secure 0 0 0) 0x7f9c74089f80 con 0x7f9c7c109110 2024-09-13T13:29:20.349 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.349+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9c6407adb0 msgr2=0x7f9c6407d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.349+0000 7f9c82642640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9c6407adb0 0x7f9c6407d270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009530 tx=0x7f9c6c009260 comp rx=0 tx=0).stop 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.349+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 msgr2=0x7f9c7c19e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.349+0000 7f9c82642640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c19e050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009b30 tx=0x7f9c74030a50 comp rx=0 tx=0).stop 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 shutdown_connections 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f9c6407adb0 0x7f9c6407d270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c109110 0x7f9c7c19e050 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c108790 0x7f9c7c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 --2- 172.21.15.43:0/3135885175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c7c102d80 0x7f9c7c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 >> 172.21.15.43:0/3135885175 conn(0x7f9c7c0fe7d0 msgr2=0x7f9c7c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 shutdown_connections 2024-09-13T13:29:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:20.350+0000 7f9c82642640 1 -- 172.21.15.43:0/3135885175 wait complete. 2024-09-13T13:29:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:20 smithi189 ceph-mon[32912]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 118 op/s 2024-09-13T13:29:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3135885175' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-13T13:29:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:20 smithi043 ceph-mon[40255]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 118 op/s 2024-09-13T13:29:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3135885175' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-13T13:29:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:20 smithi043 ceph-mon[30106]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 118 op/s 2024-09-13T13:29:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3135885175' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-13T13:29:21.041 INFO:teuthology.orchestra.run.smithi043.stdout:[{"version":23,"timestamp":"2024-09-13T13:29:09.934327+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"Pya8pnybNybQWRO0N8RGX2bdnqut8KA0HtA4Hl69"}]}] 2024-09-13T13:29:21.042 INFO:tasks.ceph_manager:config epoch is 23 2024-09-13T13:29:21.042 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T13:29:21.042 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-13T13:29:21.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-13T13:29:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3142717946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4101400 msgr2=0x7f0ba41017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3142717946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4101400 0x7f0ba41017e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009900 tx=0x7f0b9402a410 comp rx=0 tx=0).stop 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3142717946 shutdown_connections 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3142717946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4107790 0x7f0ba410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3142717946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba4106e10 0x7f0ba4107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3142717946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4101400 0x7f0ba41017e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3142717946 >> 172.21.15.43:0/3142717946 conn(0x7f0ba4078100 msgr2=0x7f0ba40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3142717946 shutdown_connections 2024-09-13T13:29:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.763+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3142717946 wait complete. 2024-09-13T13:29:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 Processor -- start 2024-09-13T13:29:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 -- start start 2024-09-13T13:29:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba4101400 0x7f0ba41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4106e10 0x7f0ba4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4107790 0x7f0ba419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba41111a0 con 0x7f0ba4107790 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4111020 con 0x7f0ba4101400 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba96f6640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba4111320 con 0x7f0ba4106e10 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba27fc640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4106e10 0x7f0ba4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba4101400 0x7f0ba41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba27fc640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4106e10 0x7f0ba4197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:33756/0 (socket says 172.21.15.43:33756) 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4107790 0x7f0ba419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.764+0000 7f0ba27fc640 1 -- 172.21.15.43:0/3227953112 learned_addr learned my addr 172.21.15.43:0/3227953112 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba2ffd640 1 -- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4106e10 msgr2=0x7f0ba4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba2ffd640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4106e10 0x7f0ba4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba2ffd640 1 -- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4107790 msgr2=0x7f0ba419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba2ffd640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4107790 0x7f0ba419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba2ffd640 1 -- 172.21.15.43:0/3227953112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba419d760 con 0x7f0ba4101400 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba27fc640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4106e10 0x7f0ba4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba37fe640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4107790 0x7f0ba419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba2ffd640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba4101400 0x7f0ba41976b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b9402aa00 tx=0x7f0b94003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0b8bfff640 1 -- 172.21.15.43:0/3227953112 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9403f070 con 0x7f0ba4101400 2024-09-13T13:29:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0b8bfff640 1 -- 172.21.15.43:0/3227953112 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0b9403bab0 con 0x7f0ba4101400 2024-09-13T13:29:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba419d9f0 con 0x7f0ba4101400 2024-09-13T13:29:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0b8bfff640 1 -- 172.21.15.43:0/3227953112 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94035030 con 0x7f0ba4101400 2024-09-13T13:29:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.765+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba41a4110 con 0x7f0ba4101400 2024-09-13T13:29:21.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.766+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba41017e0 con 0x7f0ba4101400 2024-09-13T13:29:21.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.767+0000 7f0b8bfff640 1 -- 172.21.15.43:0/3227953112 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f0b94042070 con 0x7f0ba4101400 2024-09-13T13:29:21.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.768+0000 7f0b8bfff640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f0b8c07adb0 0x7f0b8c07d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:21.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.768+0000 7f0b8bfff640 1 -- 172.21.15.43:0/3227953112 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f0b940be960 con 0x7f0ba4101400 2024-09-13T13:29:21.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.768+0000 7f0ba27fc640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f0b8c07adb0 0x7f0b8c07d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:21.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.768+0000 7f0ba27fc640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f0b8c07adb0 0x7f0b8c07d270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ba41068b0 tx=0x7f0b9800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:21.772+0000 7f0b8bfff640 1 -- 172.21.15.43:0/3227953112 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f0b94089c20 con 0x7f0ba4101400 2024-09-13T13:29:22.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.004+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0ba4109570 con 0x7f0ba4101400 2024-09-13T13:29:22.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.008+0000 7f0b8bfff640 1 -- 172.21.15.43:0/3227953112 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+197788 (secure 0 0 0) 0x7f0b9408ead0 con 0x7f0ba4101400 2024-09-13T13:29:22.012 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.010+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f0b8c07adb0 msgr2=0x7f0b8c07d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f0b8c07adb0 0x7f0b8c07d270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ba41068b0 tx=0x7f0b9800b4a0 comp rx=0 tx=0).stop 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba4101400 msgr2=0x7f0ba41976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba4101400 0x7f0ba41976b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b9402aa00 tx=0x7f0b94003b90 comp rx=0 tx=0).stop 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 shutdown_connections 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f0b8c07adb0 0x7f0b8c07d270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba4107790 0x7f0ba419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ba4106e10 0x7f0ba4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 --2- 172.21.15.43:0/3227953112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba4101400 0x7f0ba41976b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 >> 172.21.15.43:0/3227953112 conn(0x7f0ba4078100 msgr2=0x7f0ba40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:22.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 shutdown_connections 2024-09-13T13:29:22.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:22.011+0000 7f0ba96f6640 1 -- 172.21.15.43:0/3227953112 wait complete. 2024-09-13T13:29:22.749 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1020071518},{"type":"v1","addr":"172.21.15.43:6801","nonce":1020071518}]},"active_addr":"172.21.15.43:6801/1020071518","active_change":"2024-09-13T13:24:07.149748+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3220772747}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2143253161}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1467430990}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3419634510}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2290405517}]}]} 2024-09-13T13:29:22.754 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-13T13:29:22.754 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-13T13:29:22.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T13:29:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:22 smithi189 ceph-mon[32912]: pgmap v209: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-13T13:29:22.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3227953112' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-13T13:29:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:22 smithi043 ceph-mon[30106]: pgmap v209: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-13T13:29:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3227953112' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-13T13:29:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:22 smithi043 ceph-mon[40255]: pgmap v209: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-13T13:29:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3227953112' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-13T13:29:23.010 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:23.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.439+0000 7f0217968640 1 -- 172.21.15.43:0/2273782193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210109130 msgr2=0x7f021010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.439+0000 7f0217968640 1 --2- 172.21.15.43:0/2273782193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210109130 0x7f021010f930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0208009ca0 tx=0x7f020802a8c0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 -- 172.21.15.43:0/2273782193 shutdown_connections 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 --2- 172.21.15.43:0/2273782193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210109130 0x7f021010f930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 --2- 172.21.15.43:0/2273782193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02101087b0 0x7f0210108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 --2- 172.21.15.43:0/2273782193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0210102da0 0x7f0210103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 -- 172.21.15.43:0/2273782193 >> 172.21.15.43:0/2273782193 conn(0x7f02100fe810 msgr2=0x7f0210100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 -- 172.21.15.43:0/2273782193 shutdown_connections 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 -- 172.21.15.43:0/2273782193 wait complete. 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.440+0000 7f0217968640 1 Processor -- start 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0217968640 1 -- start start 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0217968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210102da0 0x7f0210198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0217968640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02101087b0 0x7f0210198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0217968640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0210109130 0x7f021019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0217968640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0210112280 con 0x7f0210102da0 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0217968640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0210112100 con 0x7f02101087b0 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0217968640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f0210112400 con 0x7f0210109130 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210102da0 0x7f0210198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215f05640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0210109130 0x7f021019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210102da0 0x7f0210198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44574/0 (socket says 172.21.15.43:44574) 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 -- 172.21.15.43:0/2167803778 learned_addr learned my addr 172.21.15.43:0/2167803778 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0214f03640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02101087b0 0x7f0210198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 -- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0210109130 msgr2=0x7f021019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0210109130 0x7f021019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 -- 172.21.15.43:0/2167803778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02101087b0 msgr2=0x7f0210198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02101087b0 0x7f0210198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.441+0000 7f0215704640 1 -- 172.21.15.43:0/2167803778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021019e820 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.442+0000 7f0215704640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210102da0 0x7f0210198770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f020c00ea60 tx=0x7f020c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.442+0000 7f02067fc640 1 -- 172.21.15.43:0/2167803778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c00cde0 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.442+0000 7f02067fc640 1 -- 172.21.15.43:0/2167803778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f020c004470 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.442+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021019eb10 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.442+0000 7f02067fc640 1 -- 172.21.15.43:0/2167803778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c010430 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.442+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101a51d0 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.443+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0210103180 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.443+0000 7f02067fc640 1 -- 172.21.15.43:0/2167803778 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f020c003f60 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.444+0000 7f02067fc640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f01fc07adb0 0x7f01fc07d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.444+0000 7f02067fc640 1 -- 172.21.15.43:0/2167803778 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f020c09e5e0 con 0x7f0210102da0 2024-09-13T13:29:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.444+0000 7f0214f03640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f01fc07adb0 0x7f01fc07d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:23.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.444+0000 7f0214f03640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f01fc07adb0 0x7f01fc07d270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02100ffba0 tx=0x7f0200005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:23.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.449+0000 7f02067fc640 1 -- 172.21.15.43:0/2167803778 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f020c0698a0 con 0x7f0210102da0 2024-09-13T13:29:23.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.636+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f021010af10 con 0x7f0210102da0 2024-09-13T13:29:23.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.637+0000 7f02067fc640 1 -- 172.21.15.43:0/2167803778 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+22771 (secure 0 0 0) 0x7f020c06e750 con 0x7f0210102da0 2024-09-13T13:29:23.639 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:29:23.639 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","created":"2024-09-13T13:23:11.853754+0000","modified":"2024-09-13T13:29:11.798297+0000","last_up_change":"2024-09-13T13:28:53.434914+0000","last_in_change":"2024-09-13T13:28:31.961750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T13:26:27.611335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-13T13:29:00.231018+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-13T13:29:01.139713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-13T13:29:02.768461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-13T13:29:04.702866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-13T13:29:06.766075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2bd16e07-e057-4c72-83d3-17490e876e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6803","nonce":2675188630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6805","nonce":2675188630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6809","nonce":2675188630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6807","nonce":2675188630}]},"public_addr":"172.21.15.43:6803/2675188630","cluster_addr":"172.21.15.43:6805/2675188630","heartbeat_back_addr":"172.21.15.43:6809/2675188630","heartbeat_front_addr":"172.21.15.43:6807/2675188630","state":["exists","up"]},{"osd":1,"uuid":"950ef5d7-8755-4a89-b1b3-953dd7f3bd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6811","nonce":966353703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6813","nonce":966353703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6817","nonce":966353703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6815","nonce":966353703}]},"public_addr":"172.21.15.43:6811/966353703","cluster_addr":"172.21.15.43:6813/966353703","heartbeat_back_addr":"172.21.15.43:6817/966353703","heartbeat_front_addr":"172.21.15.43:6815/966353703","state":["exists","up"]},{"osd":2,"uuid":"8cad8dbd-500f-46d9-9f70-27d4b12c8835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6819","nonce":601321569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6821","nonce":601321569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6825","nonce":601321569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6823","nonce":601321569}]},"public_addr":"172.21.15.43:6819/601321569","cluster_addr":"172.21.15.43:6821/601321569","heartbeat_back_addr":"172.21.15.43:6825/601321569","heartbeat_front_addr":"172.21.15.43:6823/601321569","state":["exists","up"]},{"osd":3,"uuid":"c24c101f-2a35-4ed4-8cf3-6ec3824a78de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6827","nonce":3529572742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6829","nonce":3529572742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6833","nonce":3529572742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6831","nonce":3529572742}]},"public_addr":"172.21.15.43:6827/3529572742","cluster_addr":"172.21.15.43:6829/3529572742","heartbeat_back_addr":"172.21.15.43:6833/3529572742","heartbeat_front_addr":"172.21.15.43:6831/3529572742","state":["exists","up"]},{"osd":4,"uuid":"6d500438-0ae5-4c98-bd0b-ed0955ef9aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6801","nonce":1782399195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6803","nonce":1782399195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6807","nonce":1782399195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6805","nonce":1782399195}]},"public_addr":"172.21.15.189:6801/1782399195","cluster_addr":"172.21.15.189:6803/1782399195","heartbeat_back_addr":"172.21.15.189:6807/1782399195","heartbeat_front_addr":"172.21.15.189:6805/1782399195","state":["exists","up"]},{"osd":5,"uuid":"141633af-2b7a-4224-8c9e-43d82589627b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6809","nonce":1660315521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6811","nonce":1660315521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6815","nonce":1660315521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6813","nonce":1660315521}]},"public_addr":"172.21.15.189:6809/1660315521","cluster_addr":"172.21.15.189:6811/1660315521","heartbeat_back_addr":"172.21.15.189:6815/1660315521","heartbeat_front_addr":"172.21.15.189:6813/1660315521","state":["exists","up"]},{"osd":6,"uuid":"6471db9f-5cfd-413a-8227-befdfb76cb7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6817","nonce":1554583783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6819","nonce":1554583783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6823","nonce":1554583783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6821","nonce":1554583783}]},"public_addr":"172.21.15.189:6817/1554583783","cluster_addr":"172.21.15.189:6819/1554583783","heartbeat_back_addr":"172.21.15.189:6823/1554583783","heartbeat_front_addr":"172.21.15.189:6821/1554583783","state":["exists","up"]},{"osd":7,"uuid":"b5989d1a-268c-46f0-8e74-6f8b30cb0d19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6825","nonce":1994869283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6827","nonce":1994869283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6831","nonce":1994869283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6829","nonce":1994869283}]},"public_addr":"172.21.15.189:6825/1994869283","cluster_addr":"172.21.15.189:6827/1994869283","heartbeat_back_addr":"172.21.15.189:6831/1994869283","heartbeat_front_addr":"172.21.15.189:6829/1994869283","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:25:26.648165+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:25:55.036841+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:26:25.392931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:26:54.628347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:27:22.366719+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:27:52.620989+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:28:22.633493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:28:52.273337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/85481396":"2024-09-14T13:24:07.149596+0000","172.21.15.43:6800/85481396":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/3626241883":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/1182815574":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/101460694":"2024-09-14T13:23:33.399872+0000","172.21.15.43:0/2407577026":"2024-09-14T13:23:33.399872+0000","172.21.15.43:0/1023127578":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/3370980276":"2024-09-14T13:23:33.399872+0000","172.21.15.43:6801/3041194058":"2024-09-14T13:23:33.399872+0000","172.21.15.43:6800/3041194058":"2024-09-14T13:23:33.399872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T13:29:23.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f01fc07adb0 msgr2=0x7f01fc07d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:23.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f01fc07adb0 0x7f01fc07d270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02100ffba0 tx=0x7f0200005f90 comp rx=0 tx=0).stop 2024-09-13T13:29:23.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210102da0 msgr2=0x7f0210198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:23.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210102da0 0x7f0210198770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f020c00ea60 tx=0x7f020c00ef20 comp rx=0 tx=0).stop 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 shutdown_connections 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f01fc07adb0 0x7f01fc07d270 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0210109130 0x7f021019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02101087b0 0x7f0210198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 --2- 172.21.15.43:0/2167803778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0210102da0 0x7f0210198770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 >> 172.21.15.43:0/2167803778 conn(0x7f02100fe810 msgr2=0x7f0210100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 shutdown_connections 2024-09-13T13:29:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:23.640+0000 7f0217968640 1 -- 172.21.15.43:0/2167803778 wait complete. 2024-09-13T13:29:23.681 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:23 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:23 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:23 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:24.044 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:23 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:29:24.277 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-13T13:29:24.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T13:29:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:24.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:24 smithi189 ceph-mon[32912]: pgmap v210: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.1 KiB/s wr, 98 op/s 2024-09-13T13:29:24.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2167803778' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:29:24.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:24 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:24.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:24 smithi043 ceph-mon[30106]: pgmap v210: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.1 KiB/s wr, 98 op/s 2024-09-13T13:29:24.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2167803778' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:29:24.971 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:24 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:24.972 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:24 smithi043 ceph-mon[40255]: pgmap v210: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.1 KiB/s wr, 98 op/s 2024-09-13T13:29:24.972 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2167803778' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:29:24.972 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:24 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:25.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 -- 172.21.15.43:0/452279193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec108030 msgr2=0x7f73ec10e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:25.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 --2- 172.21.15.43:0/452279193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec108030 0x7f73ec10e830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009860 tx=0x7f73e402a340 comp rx=0 tx=0).stop 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 -- 172.21.15.43:0/452279193 shutdown_connections 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 --2- 172.21.15.43:0/452279193 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec108030 0x7f73ec10e830 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 --2- 172.21.15.43:0/452279193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1076b0 0x7f73ec107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 --2- 172.21.15.43:0/452279193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec101ca0 0x7f73ec102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 -- 172.21.15.43:0/452279193 >> 172.21.15.43:0/452279193 conn(0x7f73ec0fd710 msgr2=0x7f73ec0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 -- 172.21.15.43:0/452279193 shutdown_connections 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 -- 172.21.15.43:0/452279193 wait complete. 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 Processor -- start 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.054+0000 7f73f46e0640 1 -- start start 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f46e0640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec101ca0 0x7f73ec10f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:25.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f46e0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1076b0 0x7f73ec10f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f46e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec108030 0x7f73ec10fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f46e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1122c0 con 0x7f73ec108030 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f46e0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec112140 con 0x7f73ec1076b0 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f46e0640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f73ec112440 con 0x7f73ec101ca0 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f247c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec101ca0 0x7f73ec10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec108030 0x7f73ec10fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec108030 0x7f73ec10fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44586/0 (socket says 172.21.15.43:44586) 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f247c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec101ca0 0x7f73ec10f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:33800/0 (socket says 172.21.15.43:33800) 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 -- 172.21.15.43:0/1129588069 learned_addr learned my addr 172.21.15.43:0/1129588069 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f1c7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1076b0 0x7f73ec10f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:25.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 -- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec101ca0 msgr2=0x7f73ec10f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec101ca0 0x7f73ec10f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 -- 172.21.15.43:0/1129588069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1076b0 msgr2=0x7f73ec10f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1076b0 0x7f73ec10f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 -- 172.21.15.43:0/1129588069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec1a6b50 con 0x7f73ec108030 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f247c640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec101ca0 0x7f73ec10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.055+0000 7f73f2c7d640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec108030 0x7f73ec10fdc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73e4030750 tx=0x7f73e4030780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.056+0000 7f73e37fe640 1 -- 172.21.15.43:0/1129588069 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e403f020 con 0x7f73ec108030 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.056+0000 7f73e37fe640 1 -- 172.21.15.43:0/1129588069 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f73e4004490 con 0x7f73ec108030 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.056+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec1a6de0 con 0x7f73ec108030 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.056+0000 7f73e37fe640 1 -- 172.21.15.43:0/1129588069 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4037040 con 0x7f73ec108030 2024-09-13T13:29:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.056+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a7260 con 0x7f73ec108030 2024-09-13T13:29:25.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.057+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec102080 con 0x7f73ec108030 2024-09-13T13:29:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.057+0000 7f73e37fe640 1 -- 172.21.15.43:0/1129588069 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f73e4042020 con 0x7f73ec108030 2024-09-13T13:29:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.058+0000 7f73e37fe640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f73d807adb0 0x7f73d807d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.058+0000 7f73f247c640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f73d807adb0 0x7f73d807d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.058+0000 7f73e37fe640 1 -- 172.21.15.43:0/1129588069 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f73e40bdf30 con 0x7f73ec108030 2024-09-13T13:29:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.059+0000 7f73f247c640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f73d807adb0 0x7f73d807d270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005fd0 tx=0x7f73e80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.063+0000 7f73e37fe640 1 -- 172.21.15.43:0/1129588069 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f73e4049020 con 0x7f73ec108030 2024-09-13T13:29:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.249+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73ec109e10 con 0x7f73ec108030 2024-09-13T13:29:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.250+0000 7f73e37fe640 1 -- 172.21.15.43:0/1129588069 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+22771 (secure 0 0 0) 0x7f73e40891f0 con 0x7f73ec108030 2024-09-13T13:29:25.252 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:29:25.252 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"39cb2aa4-71d3-11ef-bceb-c7b262605968","created":"2024-09-13T13:23:11.853754+0000","modified":"2024-09-13T13:29:11.798297+0000","last_up_change":"2024-09-13T13:28:53.434914+0000","last_in_change":"2024-09-13T13:28:31.961750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T13:26:27.611335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-13T13:29:00.231018+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-13T13:29:01.139713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-13T13:29:02.768461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-13T13:29:04.702866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-13T13:29:06.766075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2bd16e07-e057-4c72-83d3-17490e876e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6803","nonce":2675188630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6805","nonce":2675188630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6809","nonce":2675188630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2675188630},{"type":"v1","addr":"172.21.15.43:6807","nonce":2675188630}]},"public_addr":"172.21.15.43:6803/2675188630","cluster_addr":"172.21.15.43:6805/2675188630","heartbeat_back_addr":"172.21.15.43:6809/2675188630","heartbeat_front_addr":"172.21.15.43:6807/2675188630","state":["exists","up"]},{"osd":1,"uuid":"950ef5d7-8755-4a89-b1b3-953dd7f3bd0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6811","nonce":966353703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6813","nonce":966353703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6817","nonce":966353703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":966353703},{"type":"v1","addr":"172.21.15.43:6815","nonce":966353703}]},"public_addr":"172.21.15.43:6811/966353703","cluster_addr":"172.21.15.43:6813/966353703","heartbeat_back_addr":"172.21.15.43:6817/966353703","heartbeat_front_addr":"172.21.15.43:6815/966353703","state":["exists","up"]},{"osd":2,"uuid":"8cad8dbd-500f-46d9-9f70-27d4b12c8835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6819","nonce":601321569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6821","nonce":601321569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6825","nonce":601321569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":601321569},{"type":"v1","addr":"172.21.15.43:6823","nonce":601321569}]},"public_addr":"172.21.15.43:6819/601321569","cluster_addr":"172.21.15.43:6821/601321569","heartbeat_back_addr":"172.21.15.43:6825/601321569","heartbeat_front_addr":"172.21.15.43:6823/601321569","state":["exists","up"]},{"osd":3,"uuid":"c24c101f-2a35-4ed4-8cf3-6ec3824a78de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6827","nonce":3529572742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6829","nonce":3529572742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6833","nonce":3529572742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3529572742},{"type":"v1","addr":"172.21.15.43:6831","nonce":3529572742}]},"public_addr":"172.21.15.43:6827/3529572742","cluster_addr":"172.21.15.43:6829/3529572742","heartbeat_back_addr":"172.21.15.43:6833/3529572742","heartbeat_front_addr":"172.21.15.43:6831/3529572742","state":["exists","up"]},{"osd":4,"uuid":"6d500438-0ae5-4c98-bd0b-ed0955ef9aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6801","nonce":1782399195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6803","nonce":1782399195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6807","nonce":1782399195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1782399195},{"type":"v1","addr":"172.21.15.189:6805","nonce":1782399195}]},"public_addr":"172.21.15.189:6801/1782399195","cluster_addr":"172.21.15.189:6803/1782399195","heartbeat_back_addr":"172.21.15.189:6807/1782399195","heartbeat_front_addr":"172.21.15.189:6805/1782399195","state":["exists","up"]},{"osd":5,"uuid":"141633af-2b7a-4224-8c9e-43d82589627b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6809","nonce":1660315521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6811","nonce":1660315521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6815","nonce":1660315521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1660315521},{"type":"v1","addr":"172.21.15.189:6813","nonce":1660315521}]},"public_addr":"172.21.15.189:6809/1660315521","cluster_addr":"172.21.15.189:6811/1660315521","heartbeat_back_addr":"172.21.15.189:6815/1660315521","heartbeat_front_addr":"172.21.15.189:6813/1660315521","state":["exists","up"]},{"osd":6,"uuid":"6471db9f-5cfd-413a-8227-befdfb76cb7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6817","nonce":1554583783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6819","nonce":1554583783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6823","nonce":1554583783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1554583783},{"type":"v1","addr":"172.21.15.189:6821","nonce":1554583783}]},"public_addr":"172.21.15.189:6817/1554583783","cluster_addr":"172.21.15.189:6819/1554583783","heartbeat_back_addr":"172.21.15.189:6823/1554583783","heartbeat_front_addr":"172.21.15.189:6821/1554583783","state":["exists","up"]},{"osd":7,"uuid":"b5989d1a-268c-46f0-8e74-6f8b30cb0d19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6825","nonce":1994869283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6827","nonce":1994869283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6831","nonce":1994869283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1994869283},{"type":"v1","addr":"172.21.15.189:6829","nonce":1994869283}]},"public_addr":"172.21.15.189:6825/1994869283","cluster_addr":"172.21.15.189:6827/1994869283","heartbeat_back_addr":"172.21.15.189:6831/1994869283","heartbeat_front_addr":"172.21.15.189:6829/1994869283","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:25:26.648165+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:25:55.036841+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:26:25.392931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:26:54.628347+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:27:22.366719+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:27:52.620989+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:28:22.633493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T13:28:52.273337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/85481396":"2024-09-14T13:24:07.149596+0000","172.21.15.43:6800/85481396":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/3626241883":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/1182815574":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/101460694":"2024-09-14T13:23:33.399872+0000","172.21.15.43:0/2407577026":"2024-09-14T13:23:33.399872+0000","172.21.15.43:0/1023127578":"2024-09-14T13:24:07.149596+0000","172.21.15.43:0/3370980276":"2024-09-14T13:23:33.399872+0000","172.21.15.43:6801/3041194058":"2024-09-14T13:23:33.399872+0000","172.21.15.43:6800/3041194058":"2024-09-14T13:23:33.399872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f73d807adb0 msgr2=0x7f73d807d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f73d807adb0 0x7f73d807d270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005fd0 tx=0x7f73e80095c0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec108030 msgr2=0x7f73ec10fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec108030 0x7f73ec10fdc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73e4030750 tx=0x7f73e4030780 comp rx=0 tx=0).stop 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 shutdown_connections 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f73d807adb0 0x7f73d807d270 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec108030 0x7f73ec10fdc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1076b0 0x7f73ec10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 --2- 172.21.15.43:0/1129588069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73ec101ca0 0x7f73ec10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 >> 172.21.15.43:0/1129588069 conn(0x7f73ec0fd710 msgr2=0x7f73ec0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 shutdown_connections 2024-09-13T13:29:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:25.253+0000 7f73f46e0640 1 -- 172.21.15.43:0/1129588069 wait complete. 2024-09-13T13:29:25.793 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1129588069' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:29:25.850 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1129588069' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:29:25.850 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1129588069' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T13:29:25.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-13T13:29:25.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-13T13:29:25.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-13T13:29:25.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-13T13:29:25.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-13T13:29:25.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-13T13:29:25.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-13T13:29:25.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-13T13:29:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.467 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.471 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.471 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.471 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.476 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.476 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:26.889 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:26 smithi043 ceph-mon[30106]: pgmap v211: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1.7 KiB/s wr, 55 op/s 2024-09-13T13:29:26.889 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:26 smithi043 ceph-mon[40255]: pgmap v211: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1.7 KiB/s wr, 55 op/s 2024-09-13T13:29:26.895 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:26 smithi189 ceph-mon[32912]: pgmap v211: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1.7 KiB/s wr, 55 op/s 2024-09-13T13:29:27.148 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 systemd[1]: Starting Ceph prometheus.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:29:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 -- 172.21.15.43:0/3541802407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8107ff0 msgr2=0x7f35a810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 --2- 172.21.15.43:0/3541802407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8107ff0 0x7f35a810e7f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009ca0 tx=0x7f35a002a8c0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 -- 172.21.15.43:0/3541802407 shutdown_connections 2024-09-13T13:29:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 --2- 172.21.15.43:0/3541802407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8107ff0 0x7f35a810e7f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 --2- 172.21.15.43:0/3541802407 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35a8107670 0x7f35a8107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 --2- 172.21.15.43:0/3541802407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35a8101c60 0x7f35a8102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 -- 172.21.15.43:0/3541802407 >> 172.21.15.43:0/3541802407 conn(0x7f35a80fd6d0 msgr2=0x7f35a80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 -- 172.21.15.43:0/3541802407 shutdown_connections 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.244+0000 7f35afef7640 1 -- 172.21.15.43:0/3541802407 wait complete. 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 Processor -- start 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 -- start start 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8101c60 0x7f35a81976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35a8107670 0x7f35a8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35a8107ff0 0x7f35a819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8111180 con 0x7f35a8101c60 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8111000 con 0x7f35a8107ff0 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35afef7640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f35a8111300 con 0x7f35a8107670 2024-09-13T13:29:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35adc93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8101c60 0x7f35a81976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35ad492640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35a8107670 0x7f35a8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35adc93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8101c60 0x7f35a81976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59058/0 (socket says 172.21.15.43:59058) 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35ad492640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35a8107670 0x7f35a8197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42106/0 (socket says 172.21.15.43:42106) 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35adc93640 1 -- 172.21.15.43:0/1868220418 learned_addr learned my addr 172.21.15.43:0/1868220418 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35ae494640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35a8107ff0 0x7f35a819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35adc93640 1 -- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35a8107670 msgr2=0x7f35a8197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35adc93640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35a8107670 0x7f35a8197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35adc93640 1 -- 172.21.15.43:0/1868220418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35a8107ff0 msgr2=0x7f35a819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.245+0000 7f35adc93640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35a8107ff0 0x7f35a819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f35adc93640 1 -- 172.21.15.43:0/1868220418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a819d760 con 0x7f35a8101c60 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f35ae494640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35a8107ff0 0x7f35a819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f35ad492640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35a8107670 0x7f35a8197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:27.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f35adc93640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8101c60 0x7f35a81976b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d950 tx=0x7f359c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c017030 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f359c004020 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a819da50 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c005430 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.246+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a4110 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.247+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35a8063860 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.248+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f359c009a30 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.248+0000 7f359affd640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f359407adb0 0x7f359407d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.248+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f359c09de20 con 0x7f35a8101c60 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.248+0000 7f35ad492640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f359407adb0 0x7f359407d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.249+0000 7f359affd640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] conn(0x7f3594084880 0x7f3594086ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.249+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 --> [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f359c018ec0 con 0x7f3594084880 2024-09-13T13:29:27.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.249+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f359c0a0440 con 0x7f35a8101c60 2024-09-13T13:29:27.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.249+0000 7f35ae494640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] conn(0x7f3594084880 0x7f3594086ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.249+0000 7f35ad492640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f359407adb0 0x7f359407d270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35a40095f0 tx=0x7f35a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:27.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.249+0000 7f35ae494640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] conn(0x7f3594084880 0x7f3594086ce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:27.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.250+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== osd.1 v2:172.21.15.43:6810/966353703 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f359c018ec0 con 0x7f3594084880 2024-09-13T13:29:27.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.263+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 --> [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35a8063c60 con 0x7f3594084880 2024-09-13T13:29:27.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f359affd640 1 -- 172.21.15.43:0/1868220418 <== osd.1 v2:172.21.15.43:6810/966353703 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f35a8063c60 con 0x7f3594084880 2024-09-13T13:29:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] conn(0x7f3594084880 msgr2=0x7f3594086ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6810/966353703,v1:172.21.15.43:6811/966353703] conn(0x7f3594084880 0x7f3594086ce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f359407adb0 msgr2=0x7f359407d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f359407adb0 0x7f359407d270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35a40095f0 tx=0x7f35a4005f90 comp rx=0 tx=0).stop 2024-09-13T13:29:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8101c60 msgr2=0x7f35a81976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 --2- 172.21.15.43:0/1868220418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a8101c60 0x7f35a81976b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d950 tx=0x7f359c00de10 comp rx=0 tx=0).stop 2024-09-13T13:29:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35ae494640 1 -- 172.21.15.43:0/1868220418 reap_dead start 2024-09-13T13:29:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 shutdown_connections 2024-09-13T13:29:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 >> 172.21.15.43:0/1868220418 conn(0x7f35a80fd6d0 msgr2=0x7f35a80ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 shutdown_connections 2024-09-13T13:29:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.264+0000 7f35afef7640 1 -- 172.21.15.43:0/1868220418 wait complete. 2024-09-13T13:29:27.540 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 podman[60857]: 2024-09-13 13:29:27.271393324 +0000 UTC m=+0.095624826 container create a44705b2a1cd54e0b1b6631daebbf10a1f3f5c55cd16bbb956a71a7f70d2f846 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:29:27.541 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 podman[60857]: 2024-09-13 13:29:27.193476492 +0000 UTC m=+0.017708002 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-13T13:29:27.541 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 podman[60857]: 2024-09-13 13:29:27.515442524 +0000 UTC m=+0.339674043 container init a44705b2a1cd54e0b1b6631daebbf10a1f3f5c55cd16bbb956a71a7f70d2f846 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:29:27.541 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 podman[60857]: 2024-09-13 13:29:27.520178178 +0000 UTC m=+0.344409684 container start a44705b2a1cd54e0b1b6631daebbf10a1f3f5c55cd16bbb956a71a7f70d2f846 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:29:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 -- 172.21.15.43:0/2444096646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4e0101c60 msgr2=0x7fd4e0102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 --2- 172.21.15.43:0/2444096646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4e0101c60 0x7fd4e0102040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009860 tx=0x7fd4d002a340 comp rx=0 tx=0).stop 2024-09-13T13:29:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 -- 172.21.15.43:0/2444096646 shutdown_connections 2024-09-13T13:29:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 --2- 172.21.15.43:0/2444096646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e0107ff0 0x7fd4e010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 --2- 172.21.15.43:0/2444096646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4e0107670 0x7fd4e0107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 --2- 172.21.15.43:0/2444096646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4e0101c60 0x7fd4e0102040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 -- 172.21.15.43:0/2444096646 >> 172.21.15.43:0/2444096646 conn(0x7fd4e00fd6d0 msgr2=0x7fd4e00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 -- 172.21.15.43:0/2444096646 shutdown_connections 2024-09-13T13:29:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 -- 172.21.15.43:0/2444096646 wait complete. 2024-09-13T13:29:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 Processor -- start 2024-09-13T13:29:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.853+0000 7fd4e6005640 1 -- start start 2024-09-13T13:29:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4e6005640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e0101c60 0x7fd4e0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4e6005640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4e0107670 0x7fd4e0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4e6005640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4e0107ff0 0x7fd4e019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4e6005640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0111180 con 0x7fd4e0101c60 2024-09-13T13:29:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4e6005640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0111000 con 0x7fd4e0107ff0 2024-09-13T13:29:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4e6005640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e0111300 con 0x7fd4e0107670 2024-09-13T13:29:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e0101c60 0x7fd4e0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4d6ffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4e0107670 0x7fd4e0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e0101c60 0x7fd4e0199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59090/0 (socket says 172.21.15.43:59090) 2024-09-13T13:29:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 -- 172.21.15.43:0/2358271109 learned_addr learned my addr 172.21.15.43:0/2358271109 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4dffff640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4e0107ff0 0x7fd4e019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 -- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4e0107670 msgr2=0x7fd4e0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4e0107670 0x7fd4e0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 -- 172.21.15.43:0/2358271109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4e0107ff0 msgr2=0x7fd4e019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4e0107ff0 0x7fd4e019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 -- 172.21.15.43:0/2358271109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e01a4950 con 0x7fd4e0101c60 2024-09-13T13:29:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.854+0000 7fd4df7fe640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e0101c60 0x7fd4e0199570 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0035040 tx=0x7fd4d0033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.855+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d003f020 con 0x7fd4e0101c60 2024-09-13T13:29:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.855+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fd4d00306b0 con 0x7fd4e0101c60 2024-09-13T13:29:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.855+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0037040 con 0x7fd4e0101c60 2024-09-13T13:29:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.855+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e0197360 con 0x7fd4e0101c60 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.855+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e0197840 con 0x7fd4e0101c60 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.855+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4e00ff990 con 0x7fd4e0101c60 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.856+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7fd4d0042020 con 0x7fd4e0101c60 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4dd7fa640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd4cc07adb0 0x7fd4cc07d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7fd4d00be0e0 con 0x7fd4e0101c60 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4d6ffd640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd4cc07adb0 0x7fd4cc07d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4dd7fa640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] conn(0x7fd4cc084800 0x7fd4cc086c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 --> [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4d0030d80 con 0x7fd4cc084800 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd4d00c5050 con 0x7fd4e0101c60 2024-09-13T13:29:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4dffff640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] conn(0x7fd4cc084800 0x7fd4cc086c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.857+0000 7fd4d6ffd640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd4cc07adb0 0x7fd4cc07d270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd4e006b680 tx=0x7fd4c8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.858+0000 7fd4dffff640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] conn(0x7fd4cc084800 0x7fd4cc086c60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.858+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== osd.2 v2:172.21.15.43:6818/601321569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd4d0030d80 con 0x7fd4cc084800 2024-09-13T13:29:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 --> [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd4e0102f10 con 0x7fd4cc084800 2024-09-13T13:29:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4dd7fa640 1 -- 172.21.15.43:0/2358271109 <== osd.2 v2:172.21.15.43:6818/601321569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd4e0102f10 con 0x7fd4cc084800 2024-09-13T13:29:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] conn(0x7fd4cc084800 msgr2=0x7fd4cc086c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4e6005640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6818/601321569,v1:172.21.15.43:6819/601321569] conn(0x7fd4cc084800 0x7fd4cc086c60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd4cc07adb0 msgr2=0x7fd4cc07d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4e6005640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7fd4cc07adb0 0x7fd4cc07d270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd4e006b680 tx=0x7fd4c8010040 comp rx=0 tx=0).stop 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e0101c60 msgr2=0x7fd4e0199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4e6005640 1 --2- 172.21.15.43:0/2358271109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e0101c60 0x7fd4e0199570 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0035040 tx=0x7fd4d0033d00 comp rx=0 tx=0).stop 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.872+0000 7fd4dffff640 1 -- 172.21.15.43:0/2358271109 reap_dead start 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.873+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 shutdown_connections 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.873+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 >> 172.21.15.43:0/2358271109 conn(0x7fd4e00fd6d0 msgr2=0x7fd4e00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.873+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 shutdown_connections 2024-09-13T13:29:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.873+0000 7fd4e6005640 1 -- 172.21.15.43:0/2358271109 wait complete. 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.539Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.539Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.539Z caller=main.go:623 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi189 (none))" 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.539Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.539Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.541Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.541Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.543Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.543Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.545Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-13T13:29:27.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.545Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.659µs 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.545Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.545Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.545Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=22.223µs wal_replay_duration=165.352µs wbl_replay_duration=348ns total_replay_duration=205.836µs 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.546Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.546Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.546Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.562Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.330535ms db_storage=701ns remote_storage=1.298µs web_handler=339ns query_engine=616ns scrape=394.467µs scrape_sd=90.189µs notify=871ns notify_sd=817ns rules=14.511857ms tracing=6.643µs 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.562Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:29:27.562Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 bash[60857]: a44705b2a1cd54e0b1b6631daebbf10a1f3f5c55cd16bbb956a71a7f70d2f846 2024-09-13T13:29:27.900 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:29:27 smithi189 systemd[1]: Started Ceph prometheus.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:29:28.607 INFO:teuthology.orchestra.run.smithi043.stdout:77309411368 2024-09-13T13:29:28.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-13T13:29:28.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[40255]: pgmap v212: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-13T13:29:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-13T13:29:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[30106]: pgmap v212: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-13T13:29:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:28.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:28.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-13T13:29:28.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ignoring --setuser ceph since I am not root 2024-09-13T13:29:28.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:28 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ignoring --setgroup ceph since I am not root 2024-09-13T13:29:29.148 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: ignoring --setuser ceph since I am not root 2024-09-13T13:29:29.148 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: ignoring --setgroup ceph since I am not root 2024-09-13T13:29:29.148 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-mgr[34588]: -- 172.21.15.189:0/1500532066 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555eea7390e0 con 0x555eea717400 2024-09-13T13:29:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-mon[32912]: pgmap v212: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-13T13:29:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' 2024-09-13T13:29:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:28 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-13T13:29:29.873 INFO:teuthology.orchestra.run.smithi043.stdout:55834574892 2024-09-13T13:29:29.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-13T13:29:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:29 smithi043 ceph-mon[40255]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-13T13:29:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:29 smithi043 ceph-mon[40255]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-13T13:29:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:29 smithi043 ceph-mon[30106]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-13T13:29:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:29 smithi043 ceph-mon[30106]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-13T13:29:30.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 -- 172.21.15.43:0/3743257706 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c10d0d0 msgr2=0x7f307c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 --2- 172.21.15.43:0/3743257706 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c10d0d0 0x7f307c10f530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3074009ca0 tx=0x7f307402a8c0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 -- 172.21.15.43:0/3743257706 shutdown_connections 2024-09-13T13:29:30.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 --2- 172.21.15.43:0/3743257706 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c10d0d0 0x7f307c10f530 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 --2- 172.21.15.43:0/3743257706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f307c104590 0x7f307c10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 --2- 172.21.15.43:0/3743257706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307c103c70 0x7f307c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 -- 172.21.15.43:0/3743257706 >> 172.21.15.43:0/3743257706 conn(0x7f307c0fd6d0 msgr2=0x7f307c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 -- 172.21.15.43:0/3743257706 shutdown_connections 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.083+0000 7f3081b8c640 1 -- 172.21.15.43:0/3743257706 wait complete. 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 Processor -- start 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 -- start start 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307c103c70 0x7f307c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c104590 0x7f307c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f307c10d0d0 0x7f307c19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f307c112210 con 0x7f307c103c70 2024-09-13T13:29:30.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f307c112090 con 0x7f307c10d0d0 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f3081b8c640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f307c112390 con 0x7f307c104590 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f307affd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c104590 0x7f307c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f307affd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c104590 0x7f307c198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42156/0 (socket says 172.21.15.43:42156) 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f307affd640 1 -- 172.21.15.43:0/3853296087 learned_addr learned my addr 172.21.15.43:0/3853296087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f307b7fe640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307c103c70 0x7f307c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.084+0000 7f307bfff640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f307c10d0d0 0x7f307c19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f307affd640 1 -- 172.21.15.43:0/3853296087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f307c10d0d0 msgr2=0x7f307c19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f307affd640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f307c10d0d0 0x7f307c19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f307affd640 1 -- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307c103c70 msgr2=0x7f307c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f307affd640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307c103c70 0x7f307c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f307affd640 1 -- 172.21.15.43:0/3853296087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c19e830 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f307affd640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c104590 0x7f307c198cc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9c0 tx=0x7f306400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306400ccc0 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3064004470 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307c19eb20 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f307c1a51e0 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.085+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064003cf0 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.086+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f307c063860 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.087+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 18) ==== 103144+0+0 (secure 0 0 0) 0x7f3064003e90 con 0x7f307c104590 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f3078ff9640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f306807ae00 0x7f306807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f307b7fe640 1 -- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f306807ae00 msgr2=0x7f306807d2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1020071518 2024-09-13T13:29:30.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f306409e900 con 0x7f307c104590 2024-09-13T13:29:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f307b7fe640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f306807ae00 0x7f306807d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:29:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f3078ff9640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] conn(0x7f30680848d0 0x7f3068086d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 --> [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3068087400 con 0x7f30680848d0 2024-09-13T13:29:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f306409ede0 con 0x7f307c104590 2024-09-13T13:29:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f307bfff640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] conn(0x7f30680848d0 0x7f3068086d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.088+0000 7f307bfff640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] conn(0x7f30680848d0 0x7f3068086d30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.089+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== osd.3 v2:172.21.15.43:6826/3529572742 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3068087400 con 0x7f30680848d0 2024-09-13T13:29:30.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.103+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 --> [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f307c063c60 con 0x7f30680848d0 2024-09-13T13:29:30.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.103+0000 7f3078ff9640 1 -- 172.21.15.43:0/3853296087 <== osd.3 v2:172.21.15.43:6826/3529572742 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f307c063c60 con 0x7f30680848d0 2024-09-13T13:29:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.103+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] conn(0x7f30680848d0 msgr2=0x7f3068086d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.103+0000 7f3081b8c640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6826/3529572742,v1:172.21.15.43:6827/3529572742] conn(0x7f30680848d0 0x7f3068086d30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f306807ae00 msgr2=0x7f306807d2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:29:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f306807ae00 0x7f306807d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c104590 msgr2=0x7f307c198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 --2- 172.21.15.43:0/3853296087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f307c104590 0x7f307c198cc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9c0 tx=0x7f306400ee80 comp rx=0 tx=0).stop 2024-09-13T13:29:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f307bfff640 1 -- 172.21.15.43:0/3853296087 reap_dead start 2024-09-13T13:29:30.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 shutdown_connections 2024-09-13T13:29:30.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 >> 172.21.15.43:0/3853296087 conn(0x7f307c0fd6d0 msgr2=0x7f307c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:30.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 shutdown_connections 2024-09-13T13:29:30.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.104+0000 7f3081b8c640 1 -- 172.21.15.43:0/3853296087 wait complete. 2024-09-13T13:29:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:29 smithi189 ceph-mon[32912]: from='mgr.14152 172.21.15.43:0/3238741876' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-13T13:29:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:29 smithi189 ceph-mon[32912]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-13T13:29:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 -- 172.21.15.43:0/3942357681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0107670 msgr2=0x7f2cb0107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 --2- 172.21.15.43:0/3942357681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0107670 0x7f2cb0107ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c9802a410 comp rx=0 tx=0).stop 2024-09-13T13:29:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 -- 172.21.15.43:0/3942357681 shutdown_connections 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 --2- 172.21.15.43:0/3942357681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0107ff0 0x7f2cb010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 --2- 172.21.15.43:0/3942357681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0107670 0x7f2cb0107ab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 --2- 172.21.15.43:0/3942357681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cb0101c60 0x7f2cb0102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 -- 172.21.15.43:0/3942357681 >> 172.21.15.43:0/3942357681 conn(0x7f2cb00fd6d0 msgr2=0x7f2cb00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 -- 172.21.15.43:0/3942357681 shutdown_connections 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 -- 172.21.15.43:0/3942357681 wait complete. 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 Processor -- start 2024-09-13T13:29:30.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.649+0000 7f2cb6074640 1 -- start start 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2cb6074640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0101c60 0x7f2cb019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2cb6074640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0107670 0x7f2cb019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2cb6074640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cb0107ff0 0x7f2cb01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2cb6074640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0111110 con 0x7f2cb0107ff0 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2cb6074640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0110f90 con 0x7f2cb0107670 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2cb6074640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb0111290 con 0x7f2cb0101c60 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0101c60 0x7f2cb019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0101c60 0x7f2cb019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42184/0 (socket says 172.21.15.43:42184) 2024-09-13T13:29:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 -- 172.21.15.43:0/962954177 learned_addr learned my addr 172.21.15.43:0/962954177 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caffff640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cb0107ff0 0x7f2cb01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 -- 172.21.15.43:0/962954177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0107670 msgr2=0x7f2cb019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0107670 0x7f2cb019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 -- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cb0107ff0 msgr2=0x7f2cb01a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cb0107ff0 0x7f2cb01a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 -- 172.21.15.43:0/962954177 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb01a0a50 con 0x7f2cb0101c60 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.650+0000 7f2caf7fe640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0101c60 0x7f2cb019a9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000ea60 tx=0x7f2ca000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.651+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000cde0 con 0x7f2cb0101c60 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.651+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f2ca0004470 con 0x7f2cb0101c60 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.651+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb01a0d40 con 0x7f2cb0101c60 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.651+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0010430 con 0x7f2cb0101c60 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.651+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb01a75d0 con 0x7f2cb0101c60 2024-09-13T13:29:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.652+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2cb00fed90 con 0x7f2cb0101c60 2024-09-13T13:29:30.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.652+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 18) ==== 103144+0+0 (secure 0 0 0) 0x7f2ca0003f60 con 0x7f2cb0101c60 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2cad7fa640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2c9c07ae00 0x7f2c9c07d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2ca6ffd640 1 -- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2c9c07ae00 msgr2=0x7f2c9c07d2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1020071518 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f2ca009e610 con 0x7f2cb0101c60 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2ca6ffd640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2c9c07ae00 0x7f2c9c07d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2cad7fa640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] conn(0x7f2c9c084950 0x7f2c9c086db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 --> [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c9c087480 con 0x7f2c9c084950 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f2ca00a5050 con 0x7f2cb0101c60 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.653+0000 7f2caffff640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] conn(0x7f2c9c084950 0x7f2c9c086db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.654+0000 7f2caffff640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] conn(0x7f2c9c084950 0x7f2c9c086db0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:30.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.655+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== osd.5 v2:172.21.15.189:6808/1660315521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c9c087480 con 0x7f2c9c084950 2024-09-13T13:29:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.668+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 --> [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2cb0102f10 con 0x7f2c9c084950 2024-09-13T13:29:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cad7fa640 1 -- 172.21.15.43:0/962954177 <== osd.5 v2:172.21.15.189:6808/1660315521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2cb0102f10 con 0x7f2c9c084950 2024-09-13T13:29:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 >> [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] conn(0x7f2c9c084950 msgr2=0x7f2c9c086db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.189:6808/1660315521,v1:172.21.15.189:6809/1660315521] conn(0x7f2c9c084950 0x7f2c9c086db0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2c9c07ae00 msgr2=0x7f2c9c07d2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f2c9c07ae00 0x7f2c9c07d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0101c60 msgr2=0x7f2cb019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 --2- 172.21.15.43:0/962954177 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2cb0101c60 0x7f2cb019a9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000ea60 tx=0x7f2ca000ef20 comp rx=0 tx=0).stop 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2caffff640 1 -- 172.21.15.43:0/962954177 reap_dead start 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 shutdown_connections 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 >> 172.21.15.43:0/962954177 conn(0x7f2cb00fd6d0 msgr2=0x7f2cb006aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 shutdown_connections 2024-09-13T13:29:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:30.669+0000 7f2cb6074640 1 -- 172.21.15.43:0/962954177 wait complete. 2024-09-13T13:29:31.643 INFO:teuthology.orchestra.run.smithi043.stdout:158913789973 2024-09-13T13:29:31.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-13T13:29:31.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 -- 172.21.15.43:0/135546596 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec107670 msgr2=0x7f38ec107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:31.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 --2- 172.21.15.43:0/135546596 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec107670 0x7f38ec107ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009860 tx=0x7f38dc02a340 comp rx=0 tx=0).stop 2024-09-13T13:29:31.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 -- 172.21.15.43:0/135546596 shutdown_connections 2024-09-13T13:29:31.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 --2- 172.21.15.43:0/135546596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec107ff0 0x7f38ec10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:31.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 --2- 172.21.15.43:0/135546596 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec107670 0x7f38ec107ab0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:31.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 --2- 172.21.15.43:0/135546596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38ec101c60 0x7f38ec102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:31.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 -- 172.21.15.43:0/135546596 >> 172.21.15.43:0/135546596 conn(0x7f38ec0fd6d0 msgr2=0x7f38ec0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 -- 172.21.15.43:0/135546596 shutdown_connections 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.175+0000 7f38f4548640 1 -- 172.21.15.43:0/135546596 wait complete. 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 Processor -- start 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 -- start start 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec101c60 0x7f38ec0ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec107670 0x7f38ec0fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38ec107ff0 0x7f38ec0fe200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec111180 con 0x7f38ec107670 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec111000 con 0x7f38ec107ff0 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f4548640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec111300 con 0x7f38ec101c60 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f22e4640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec101c60 0x7f38ec0ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f22e4640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec101c60 0x7f38ec0ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42100/0 (socket says 172.21.15.43:42100) 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f1ae3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec107670 0x7f38ec0fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f22e4640 1 -- 172.21.15.43:0/3837481949 learned_addr learned my addr 172.21.15.43:0/3837481949 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f2ae5640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38ec107ff0 0x7f38ec0fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f1ae3640 1 -- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec101c60 msgr2=0x7f38ec0ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f1ae3640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec101c60 0x7f38ec0ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f1ae3640 1 -- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38ec107ff0 msgr2=0x7f38ec0fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.176+0000 7f38f1ae3640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38ec107ff0 0x7f38ec0fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38f1ae3640 1 -- 172.21.15.43:0/3837481949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec1ab130 con 0x7f38ec107670 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38f22e4640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38ec101c60 0x7f38ec0ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38f1ae3640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec107670 0x7f38ec0fdbd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38dc02a860 tx=0x7f38dc02aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc037040 con 0x7f38ec107670 2024-09-13T13:29:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec1ab300 con 0x7f38ec107670 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f38dc004050 con 0x7f38ec107670 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1ab7b0 con 0x7f38ec107670 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.177+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc030e80 con 0x7f38ec107670 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.178+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38ec063860 con 0x7f38ec107670 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.179+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 103130+0+0 (secure 0 0 0) 0x7f38dc042020 con 0x7f38ec107670 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38e37fe640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f38d807af50 0x7f38d807d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38f22e4640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f38d807af50 0x7f38d807d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f38dc0bdde0 con 0x7f38ec107670 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38e37fe640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] conn(0x7f38d8084a20 0x7f38d8086e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 --> [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38dc030e80 con 0x7f38d8084a20 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f38dc0be2c0 con 0x7f38ec107670 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38f22e4640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f38d807af50 0x7f38d807d410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f38e8011550 tx=0x7f38e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.180+0000 7f38f2ae5640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] conn(0x7f38d8084a20 0x7f38d8086e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.181+0000 7f38f2ae5640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] conn(0x7f38d8084a20 0x7f38d8086e80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.181+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== osd.6 v2:172.21.15.189:6816/1554583783 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38dc030e80 con 0x7f38d8084a20 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.195+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 --> [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f38ec063c60 con 0x7f38d8084a20 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38e37fe640 1 -- 172.21.15.43:0/3837481949 <== osd.6 v2:172.21.15.189:6816/1554583783 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f38ec063c60 con 0x7f38d8084a20 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] conn(0x7f38d8084a20 msgr2=0x7f38d8086e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.189:6816/1554583783,v1:172.21.15.189:6817/1554583783] conn(0x7f38d8084a20 0x7f38d8086e80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f38d807af50 msgr2=0x7f38d807d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f38d807af50 0x7f38d807d410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f38e8011550 tx=0x7f38e8005f90 comp rx=0 tx=0).stop 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec107670 msgr2=0x7f38ec0fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 --2- 172.21.15.43:0/3837481949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec107670 0x7f38ec0fdbd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38dc02a860 tx=0x7f38dc02aeb0 comp rx=0 tx=0).stop 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f2ae5640 1 -- 172.21.15.43:0/3837481949 reap_dead start 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 shutdown_connections 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 >> 172.21.15.43:0/3837481949 conn(0x7f38ec0fd6d0 msgr2=0x7f38ec06ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 shutdown_connections 2024-09-13T13:29:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:27.196+0000 7f38f4548640 1 -- 172.21.15.43:0/3837481949 wait complete. 2024-09-13T13:29:32.407 INFO:teuthology.orchestra.run.smithi043.stdout:184683593743 2024-09-13T13:29:32.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-13T13:29:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 -- 172.21.15.43:0/1200463476 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1100108010 msgr2=0x7f110010e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 --2- 172.21.15.43:0/1200463476 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1100108010 0x7f110010e810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009ca0 tx=0x7f10f802a8c0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 -- 172.21.15.43:0/1200463476 shutdown_connections 2024-09-13T13:29:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 --2- 172.21.15.43:0/1200463476 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1100108010 0x7f110010e810 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 --2- 172.21.15.43:0/1200463476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1100107690 0x7f1100107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 --2- 172.21.15.43:0/1200463476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1100101c80 0x7f1100102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 -- 172.21.15.43:0/1200463476 >> 172.21.15.43:0/1200463476 conn(0x7f11000fd6d0 msgr2=0x7f11000ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 -- 172.21.15.43:0/1200463476 shutdown_connections 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.618+0000 7f11055b2640 1 -- 172.21.15.43:0/1200463476 wait complete. 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.619+0000 7f11055b2640 1 Processor -- start 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f11055b2640 1 -- start start 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f11055b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1100101c80 0x7f110019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f11055b2640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1100107690 0x7f110019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f11055b2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1100108010 0x7f11001a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f11055b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11001100b0 con 0x7f1100101c80 2024-09-13T13:29:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f11055b2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f110010ff30 con 0x7f1100108010 2024-09-13T13:29:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f11055b2640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f1100110230 con 0x7f1100107690 2024-09-13T13:29:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10ff7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1100108010 0x7f11001a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1100101c80 0x7f110019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10ff7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1100108010 0x7f11001a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.43:59438/0 (socket says 172.21.15.43:59438) 2024-09-13T13:29:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10ff7fe640 1 -- 172.21.15.43:0/2169594831 learned_addr learned my addr 172.21.15.43:0/2169594831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10fe7fc640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1100107690 0x7f110019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10feffd640 1 -- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1100107690 msgr2=0x7f110019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10feffd640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1100107690 0x7f110019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10feffd640 1 -- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1100108010 msgr2=0x7f11001a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10feffd640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1100108010 0x7f11001a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10feffd640 1 -- 172.21.15.43:0/2169594831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11001a0a80 con 0x7f1100101c80 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.620+0000 7f10ff7fe640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1100108010 0x7f11001a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.621+0000 7f10feffd640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1100101c80 0x7f110019aa20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097d0 tx=0x7f10f0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.621+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f000e8a0 con 0x7f1100101c80 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.621+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11001a0d70 con 0x7f1100101c80 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.621+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f10f0019070 con 0x7f1100101c80 2024-09-13T13:29:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.621+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0014d00 con 0x7f1100101c80 2024-09-13T13:29:32.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.621+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11001a7600 con 0x7f1100101c80 2024-09-13T13:29:32.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.622+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1100063820 con 0x7f1100101c80 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.623+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 103144+0+0 (secure 0 0 0) 0x7f10f000ea40 con 0x7f1100101c80 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.624+0000 7f10e7fff640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10e807ae00 0x7f10e807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.624+0000 7f10fe7fc640 1 -- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10e807ae00 msgr2=0x7f10e807d2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1020071518 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.624+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f10f009ea00 con 0x7f1100101c80 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.624+0000 7f10fe7fc640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10e807ae00 0x7f10e807d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.624+0000 7f10e7fff640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] conn(0x7f10e8084950 0x7f10e8086db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.624+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 --> [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10e8087480 con 0x7f10e8084950 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.624+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f10f009eee0 con 0x7f1100101c80 2024-09-13T13:29:32.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.625+0000 7f10ff7fe640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] conn(0x7f10e8084950 0x7f10e8086db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.625+0000 7f10ff7fe640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] conn(0x7f10e8084950 0x7f10e8086db0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:32.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.626+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== osd.4 v2:172.21.15.189:6800/1782399195 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10e8087480 con 0x7f10e8084950 2024-09-13T13:29:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.641+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 --> [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1100063c20 con 0x7f10e8084950 2024-09-13T13:29:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f10e7fff640 1 -- 172.21.15.43:0/2169594831 <== osd.4 v2:172.21.15.189:6800/1782399195 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1100063c20 con 0x7f10e8084950 2024-09-13T13:29:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] conn(0x7f10e8084950 msgr2=0x7f10e8086db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.189:6800/1782399195,v1:172.21.15.189:6801/1782399195] conn(0x7f10e8084950 0x7f10e8086db0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10e807ae00 msgr2=0x7f10e807d2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:29:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f10e807ae00 0x7f10e807d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1100101c80 msgr2=0x7f110019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 --2- 172.21.15.43:0/2169594831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1100101c80 0x7f110019aa20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097d0 tx=0x7f10f0009c90 comp rx=0 tx=0).stop 2024-09-13T13:29:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f10ff7fe640 1 -- 172.21.15.43:0/2169594831 reap_dead start 2024-09-13T13:29:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 shutdown_connections 2024-09-13T13:29:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.642+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 >> 172.21.15.43:0/2169594831 conn(0x7f11000fd6d0 msgr2=0x7f1100104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.643+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 shutdown_connections 2024-09-13T13:29:32.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.643+0000 7f11055b2640 1 -- 172.21.15.43:0/2169594831 wait complete. 2024-09-13T13:29:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 -- 172.21.15.43:0/3275278655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94101c60 msgr2=0x7f6d94102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 --2- 172.21.15.43:0/3275278655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94101c60 0x7f6d94102040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009900 tx=0x7f6d8402a410 comp rx=0 tx=0).stop 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 -- 172.21.15.43:0/3275278655 shutdown_connections 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 --2- 172.21.15.43:0/3275278655 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d94107ff0 0x7f6d9410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 --2- 172.21.15.43:0/3275278655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d94107670 0x7f6d94107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 --2- 172.21.15.43:0/3275278655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94101c60 0x7f6d94102040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 -- 172.21.15.43:0/3275278655 >> 172.21.15.43:0/3275278655 conn(0x7f6d940fd6d0 msgr2=0x7f6d940ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 -- 172.21.15.43:0/3275278655 shutdown_connections 2024-09-13T13:29:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.952+0000 7f6d99e71640 1 -- 172.21.15.43:0/3275278655 wait complete. 2024-09-13T13:29:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.953+0000 7f6d99e71640 1 Processor -- start 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.953+0000 7f6d99e71640 1 -- start start 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d99e71640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d94101c60 0x7f6d94199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d99e71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94107670 0x7f6d94199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d99e71640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d94107ff0 0x7f6d941a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d99e71640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94111180 con 0x7f6d94107670 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d99e71640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94111000 con 0x7f6d94107ff0 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d99e71640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f6d94111300 con 0x7f6d94101c60 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94107670 0x7f6d94199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d937fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d94101c60 0x7f6d94199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94107670 0x7f6d94199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59176/0 (socket says 172.21.15.43:59176) 2024-09-13T13:29:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 -- 172.21.15.43:0/4274911949 learned_addr learned my addr 172.21.15.43:0/4274911949 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d93fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d94107ff0 0x7f6d941a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 -- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d94101c60 msgr2=0x7f6d94199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d94101c60 0x7f6d94199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 -- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d94107ff0 msgr2=0x7f6d941a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d94107ff0 0x7f6d941a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d92ffd640 1 -- 172.21.15.43:0/4274911949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d941a4b30 con 0x7f6d94107670 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.954+0000 7f6d93fff640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d94107ff0 0x7f6d941a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.955+0000 7f6d92ffd640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94107670 0x7f6d94199ab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800ea60 tx=0x7f6d8800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.955+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800cd20 con 0x7f6d94107670 2024-09-13T13:29:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.955+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d941a4d00 con 0x7f6d94107670 2024-09-13T13:29:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.955+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6d88004470 con 0x7f6d94107670 2024-09-13T13:29:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.955+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003e60 con 0x7f6d94107670 2024-09-13T13:29:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.955+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d941a51d0 con 0x7f6d94107670 2024-09-13T13:29:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.956+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d94063860 con 0x7f6d94107670 2024-09-13T13:29:32.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.957+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 103144+0+0 (secure 0 0 0) 0x7f6d88004000 con 0x7f6d94107670 2024-09-13T13:29:32.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.958+0000 7f6d90ff9640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6d7c07ae00 0x7f6d7c07d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.958+0000 7f6d937fe640 1 -- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6d7c07ae00 msgr2=0x7f6d7c07d2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1020071518 2024-09-13T13:29:32.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.958+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f6d8809eb40 con 0x7f6d94107670 2024-09-13T13:29:32.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.958+0000 7f6d937fe640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6d7c07ae00 0x7f6d7c07d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T13:29:32.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.958+0000 7f6d90ff9640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] conn(0x7f6d7c084950 0x7f6d7c086db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:32.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.958+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 --> [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d88016ec0 con 0x7f6d7c084950 2024-09-13T13:29:32.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.958+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f6d8809f020 con 0x7f6d94107670 2024-09-13T13:29:32.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.959+0000 7f6d93fff640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] conn(0x7f6d7c084950 0x7f6d7c086db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:32.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.959+0000 7f6d93fff640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] conn(0x7f6d7c084950 0x7f6d7c086db0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:32.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.960+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== osd.7 v2:172.21.15.189:6824/1994869283 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d88016ec0 con 0x7f6d7c084950 2024-09-13T13:29:32.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.973+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 --> [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d94063c60 con 0x7f6d7c084950 2024-09-13T13:29:32.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d90ff9640 1 -- 172.21.15.43:0/4274911949 <== osd.7 v2:172.21.15.189:6824/1994869283 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d94063c60 con 0x7f6d7c084950 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] conn(0x7f6d7c084950 msgr2=0x7f6d7c086db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.189:6824/1994869283,v1:172.21.15.189:6825/1994869283] conn(0x7f6d7c084950 0x7f6d7c086db0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6d7c07ae00 msgr2=0x7f6d7c07d2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:6800/1020071518,v1:172.21.15.43:6801/1020071518] conn(0x7f6d7c07ae00 0x7f6d7c07d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94107670 msgr2=0x7f6d94199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 --2- 172.21.15.43:0/4274911949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d94107670 0x7f6d94199ab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800ea60 tx=0x7f6d8800ee60 comp rx=0 tx=0).stop 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d93fff640 1 -- 172.21.15.43:0/4274911949 reap_dead start 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 shutdown_connections 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 >> 172.21.15.43:0/4274911949 conn(0x7f6d940fd6d0 msgr2=0x7f6d940ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.974+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 shutdown_connections 2024-09-13T13:29:32.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:32.975+0000 7f6d99e71640 1 -- 172.21.15.43:0/4274911949 wait complete. 2024-09-13T13:29:33.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-mon[32912]: Standby manager daemon x restarted 2024-09-13T13:29:33.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-mon[32912]: Standby manager daemon x started 2024-09-13T13:29:33.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T13:29:33.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-mon[32912]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: [13/Sep/2024:13:29:33] ENGINE Bus STARTING 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: CherryPy Checker: 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: The Application mounted at '' has an empty config. 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: [13/Sep/2024:13:29:33] ENGINE Serving on http://:::9283 2024-09-13T13:29:33.900 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:29:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x[34565]: [13/Sep/2024:13:29:33] ENGINE Bus STARTED 2024-09-13T13:29:33.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[30106]: Standby manager daemon x restarted 2024-09-13T13:29:33.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[30106]: Standby manager daemon x started 2024-09-13T13:29:33.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[30106]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[40255]: Standby manager daemon x restarted 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[40255]: Standby manager daemon x started 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T13:29:33.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:33 smithi043 ceph-mon[40255]: from='mgr.? 172.21.15.189:0/1369784562' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T13:29:33.961 INFO:teuthology.orchestra.run.smithi043.stdout:111669149730 2024-09-13T13:29:33.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-13T13:29:34.381 INFO:teuthology.orchestra.run.smithi043.stdout:210453397514 2024-09-13T13:29:34.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-13T13:29:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 -- 172.21.15.43:0/4100960134 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 msgr2=0x7f977010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 --2- 172.21.15.43:0/4100960134 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977010e7f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9768009ca0 tx=0x7f976802a8c0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 -- 172.21.15.43:0/4100960134 shutdown_connections 2024-09-13T13:29:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 --2- 172.21.15.43:0/4100960134 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977010e7f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 --2- 172.21.15.43:0/4100960134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770107670 0x7f9770107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 --2- 172.21.15.43:0/4100960134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9770101c60 0x7f9770102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 -- 172.21.15.43:0/4100960134 >> 172.21.15.43:0/4100960134 conn(0x7f97700fd6d0 msgr2=0x7f97700ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 -- 172.21.15.43:0/4100960134 shutdown_connections 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.523+0000 7f9775c39640 1 -- 172.21.15.43:0/4100960134 wait complete. 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.524+0000 7f9775c39640 1 Processor -- start 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.524+0000 7f9775c39640 1 -- start start 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f9775c39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770101c60 0x7f97701995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f9775c39640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9770107670 0x7f9770199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f9775c39640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f9775c39640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9770111180 con 0x7f9770101c60 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f9775c39640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9770111000 con 0x7f9770107670 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f9775c39640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f9770111300 con 0x7f9770107ff0 2024-09-13T13:29:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:42292/0 (socket says 172.21.15.43:42292) 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 -- 172.21.15.43:0/628767698 learned_addr learned my addr 172.21.15.43:0/628767698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976f7fe640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770101c60 0x7f97701995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976effd640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9770107670 0x7f9770199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 -- 172.21.15.43:0/628767698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9770107670 msgr2=0x7f9770199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9770107670 0x7f9770199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 -- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770101c60 msgr2=0x7f97701995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770101c60 0x7f97701995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976ffff640 1 -- 172.21.15.43:0/628767698 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97701a49d0 con 0x7f9770107ff0 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976f7fe640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770101c60 0x7f97701995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.525+0000 7f976effd640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9770107670 0x7f9770199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.526+0000 7f976ffff640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977019a090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97680097b0 tx=0x7f9768003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.526+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768003e40 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.526+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9768004490 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.526+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97701a4c60 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.526+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768030a10 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.526+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97701a51a0 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.527+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9770063820 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.528+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 20) ==== 102761+0+0 (secure 0 0 0) 0x7f9768004040 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.529+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7f97680befa0 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.529+0000 7f976cff9640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] conn(0x7f9758081c40 0x7f9758084080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.529+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 --> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9758084750 con 0x7f9758081c40 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.529+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f97680c6050 con 0x7f9770107ff0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.529+0000 7f976f7fe640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] conn(0x7f9758081c40 0x7f9758084080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.529+0000 7f976f7fe640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] conn(0x7f9758081c40 0x7f9758084080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.531+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== osd.0 v2:172.21.15.43:6802/2675188630 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9758084750 con 0x7f9758081c40 2024-09-13T13:29:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 --> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9770063c20 con 0x7f9758081c40 2024-09-13T13:29:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f976cff9640 1 -- 172.21.15.43:0/628767698 <== osd.0 v2:172.21.15.43:6802/2675188630 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9770063c20 con 0x7f9758081c40 2024-09-13T13:29:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] conn(0x7f9758081c40 msgr2=0x7f9758084080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] conn(0x7f9758081c40 0x7f9758084080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 msgr2=0x7f977019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977019a090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97680097b0 tx=0x7f9768003c20 comp rx=0 tx=0).stop 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 shutdown_connections 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:6802/2675188630,v1:172.21.15.43:6803/2675188630] conn(0x7f9758081c40 0x7f9758084080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9770107ff0 0x7f977019a090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9770107670 0x7f9770199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 --2- 172.21.15.43:0/628767698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770101c60 0x7f97701995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.545+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 >> 172.21.15.43:0/628767698 conn(0x7f97700fd6d0 msgr2=0x7f97700ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.546+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 shutdown_connections 2024-09-13T13:29:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:34.546+0000 7f9775c39640 1 -- 172.21.15.43:0/628767698 wait complete. 2024-09-13T13:29:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-13T13:29:34.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: Active manager daemon y restarted 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: Activating manager daemon y 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: osdmap e63: 8 total, 8 up, 8 in 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: mgrmap e20: y(active, starting, since 0.0102865s), standbys: x 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:29:34.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: Manager daemon y is now available 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:29:34.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: Active manager daemon y restarted 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: Activating manager daemon y 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: osdmap e63: 8 total, 8 up, 8 in 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: mgrmap e20: y(active, starting, since 0.0102865s), standbys: x 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:29:34.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T13:29:34.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: Manager daemon y is now available 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:29:34.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:34 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:29:35.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-13T13:29:35.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: Active manager daemon y restarted 2024-09-13T13:29:35.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: Activating manager daemon y 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: osdmap e63: 8 total, 8 up, 8 in 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: mgrmap e20: y(active, starting, since 0.0102865s), standbys: x 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T13:29:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: Manager daemon y is now available 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:29:35.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:29:35.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T13:29:35.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:29:35.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:34 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T13:29:35.246 INFO:teuthology.orchestra.run.smithi043.stdout:133143986203 2024-09-13T13:29:35.247 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-13T13:29:35.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:35] ENGINE Bus STARTING 2024-09-13T13:29:35.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: CherryPy Checker: 2024-09-13T13:29:35.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: The Application mounted at '' has an empty config. 2024-09-13T13:29:35.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:29:35.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:35] ENGINE Serving on http://:::9283 2024-09-13T13:29:35.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:35] ENGINE Bus STARTED 2024-09-13T13:29:35.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:29:35.674 INFO:teuthology.orchestra.run.smithi043.stdout:34359738417 2024-09-13T13:29:35.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-13T13:29:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:35.765 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:35.897 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:29:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:29:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[30106]: mgrmap e21: y(active, since 1.0139s), standbys: x 2024-09-13T13:29:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:29:36.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:36.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[30106]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:36.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[40255]: mgrmap e21: y(active, since 1.0139s), standbys: x 2024-09-13T13:29:36.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:29:36.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:36.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-mon[40255]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:36.485 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:29:36.318+0000 7fc8e2919640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:13:29:36] ENGINE Error in HTTPServer.serve 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: Traceback (most recent call last): 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._connections.run(self.expiration_interval) 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._run(expiration_interval) 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: s = self.context.wrap_socket( 2024-09-13T13:29:36.486 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:29:36.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: return self.sslsocket_class._create( 2024-09-13T13:29:36.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:29:36.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self.do_handshake() 2024-09-13T13:29:36.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:29:36.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: self._sslobj.do_handshake() 2024-09-13T13:29:36.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:29:36.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:36 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:29:36.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:36 smithi189 ceph-mon[32912]: mgrmap e21: y(active, since 1.0139s), standbys: x 2024-09-13T13:29:36.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:36 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:29:36.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:36.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:36 smithi189 ceph-mon[32912]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 -- 172.21.15.43:0/981098771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc194104500 msgr2=0x7fc19410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 --2- 172.21.15.43:0/981098771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc194104500 0x7fc19410cae0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc188009860 tx=0x7fc18802a340 comp rx=0 tx=0).stop 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 -- 172.21.15.43:0/981098771 shutdown_connections 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 --2- 172.21.15.43:0/981098771 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc19410d0b0 0x7fc19410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 --2- 172.21.15.43:0/981098771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc194104500 0x7fc19410cae0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 --2- 172.21.15.43:0/981098771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc194103b00 0x7fc194103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 -- 172.21.15.43:0/981098771 >> 172.21.15.43:0/981098771 conn(0x7fc1940fd6d0 msgr2=0x7fc1940ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 -- 172.21.15.43:0/981098771 shutdown_connections 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.923+0000 7fc19afe1640 1 -- 172.21.15.43:0/981098771 wait complete. 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 Processor -- start 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 -- start start 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc194103b00 0x7fc1941987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc194104500 0x7fc194198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19410d0b0 0x7fc19419e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc194112200 con 0x7fc19410d0b0 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc194112080 con 0x7fc194104500 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19afe1640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fc194112380 con 0x7fc194103b00 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19957e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19410d0b0 0x7fc19419e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc198d7d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc194103b00 0x7fc1941987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19957e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19410d0b0 0x7fc19419e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44286/0 (socket says 172.21.15.43:44286) 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19957e640 1 -- 172.21.15.43:0/977954341 learned_addr learned my addr 172.21.15.43:0/977954341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc193fff640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc194104500 0x7fc194198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc198d7d640 1 -- 172.21.15.43:0/977954341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc194104500 msgr2=0x7fc194198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc198d7d640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc194104500 0x7fc194198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc198d7d640 1 -- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19410d0b0 msgr2=0x7fc19419e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc198d7d640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19410d0b0 0x7fc19419e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc198d7d640 1 -- 172.21.15.43:0/977954341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19419e860 con 0x7fc194103b00 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.924+0000 7fc19957e640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19410d0b0 0x7fc19419e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.925+0000 7fc198d7d640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc194103b00 0x7fc1941987b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc18400dab0 tx=0x7fc18400deb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.925+0000 7fc191ffb640 1 -- 172.21.15.43:0/977954341 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400bd70 con 0x7fc194103b00 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.925+0000 7fc191ffb640 1 -- 172.21.15.43:0/977954341 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc184003dc0 con 0x7fc194103b00 2024-09-13T13:29:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.925+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19419eb50 con 0x7fc194103b00 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.925+0000 7fc191ffb640 1 -- 172.21.15.43:0/977954341 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184010460 con 0x7fc194103b00 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.925+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a5210 con 0x7fc194103b00 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.926+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc194103ee0 con 0x7fc194103b00 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.927+0000 7fc191ffb640 1 -- 172.21.15.43:0/977954341 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) ==== 103080+0+0 (secure 0 0 0) 0x7fc184003970 con 0x7fc194103b00 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.928+0000 7fc191ffb640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fc17c07ad80 0x7fc17c07d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.928+0000 7fc191ffb640 1 -- 172.21.15.43:0/977954341 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7fc18409e6f0 con 0x7fc194103b00 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.928+0000 7fc193fff640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fc17c07ad80 0x7fc17c07d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.928+0000 7fc193fff640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fc17c07ad80 0x7fc17c07d240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc18802a930 tx=0x7fc1880056d0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:36.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.932+0000 7fc191ffb640 1 -- 172.21.15.43:0/977954341 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc184069a80 con 0x7fc194103b00 2024-09-13T13:29:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.848+0000 7f073439b640 1 -- 172.21.15.43:0/3938476734 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 msgr2=0x7f072c10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.848+0000 7f073439b640 1 --2- 172.21.15.43:0/3938476734 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 0x7f072c10f910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0724009ca0 tx=0x7f072402a8c0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.848+0000 7f073439b640 1 -- 172.21.15.43:0/3938476734 shutdown_connections 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.848+0000 7f073439b640 1 --2- 172.21.15.43:0/3938476734 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 0x7f072c10f910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.848+0000 7f073439b640 1 --2- 172.21.15.43:0/3938476734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c108790 0x7f072c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.848+0000 7f073439b640 1 --2- 172.21.15.43:0/3938476734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f072c102d80 0x7f072c103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.848+0000 7f073439b640 1 -- 172.21.15.43:0/3938476734 >> 172.21.15.43:0/3938476734 conn(0x7f072c0fe7d0 msgr2=0x7f072c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.849+0000 7f073439b640 1 -- 172.21.15.43:0/3938476734 shutdown_connections 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.849+0000 7f073439b640 1 -- 172.21.15.43:0/3938476734 wait complete. 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.849+0000 7f073439b640 1 Processor -- start 2024-09-13T13:29:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.849+0000 7f073439b640 1 -- start start 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f073439b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c102d80 0x7f072c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f073439b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f072c108790 0x7f072c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f073439b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 0x7f072c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f073439b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f072c1122a0 con 0x7f072c102d80 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f073439b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f072c112120 con 0x7f072c108790 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f073439b640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f072c112420 con 0x7f072c109110 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732137640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c102d80 0x7f072c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732137640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c102d80 0x7f072c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44266/0 (socket says 172.21.15.43:44266) 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732137640 1 -- 172.21.15.43:0/467361911 learned_addr learned my addr 172.21.15.43:0/467361911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732938640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 0x7f072c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0731936640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f072c108790 0x7f072c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732938640 1 -- 172.21.15.43:0/467361911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f072c108790 msgr2=0x7f072c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732938640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f072c108790 0x7f072c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732938640 1 -- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c102d80 msgr2=0x7f072c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732938640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c102d80 0x7f072c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732938640 1 -- 172.21.15.43:0/467361911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c19e8a0 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732137640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c102d80 0x7f072c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.850+0000 7f0732938640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 0x7f072c19e0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f072402ade0 tx=0x7f0724003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.851+0000 7f071f7fe640 1 -- 172.21.15.43:0/467361911 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724004040 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.851+0000 7f071f7fe640 1 -- 172.21.15.43:0/467361911 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0724040070 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.851+0000 7f071f7fe640 1 -- 172.21.15.43:0/467361911 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724030b20 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.851+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c19eb30 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.851+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c1a5250 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.852+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072c103160 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.853+0000 7f071f7fe640 1 -- 172.21.15.43:0/467361911 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) ==== 103080+0+0 (secure 0 0 0) 0x7f0724030cc0 con 0x7f072c109110 2024-09-13T13:29:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.854+0000 7f071f7fe640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f071807ad80 0x7f071807d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.854+0000 7f071f7fe640 1 -- 172.21.15.43:0/467361911 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7f07240bf250 con 0x7f072c109110 2024-09-13T13:29:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.854+0000 7f0732137640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f071807ad80 0x7f071807d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.854+0000 7f0732137640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f071807ad80 0x7f071807d240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f07200104e0 tx=0x7f072000b410 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:36.859+0000 7f071f7fe640 1 -- 172.21.15.43:0/467361911 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f072408a6d0 con 0x7f072c109110 2024-09-13T13:29:37.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.048+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f072c10aef0 con 0x7f072c109110 2024-09-13T13:29:37.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.048+0000 7f071f7fe640 1 -- 172.21.15.43:0/467361911 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f072408f580 con 0x7f072c109110 2024-09-13T13:29:37.050 INFO:teuthology.orchestra.run.smithi043.stdout:77309411369 2024-09-13T13:29:37.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.050+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f071807ad80 msgr2=0x7f071807d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.050+0000 7f073439b640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f071807ad80 0x7f071807d240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f07200104e0 tx=0x7f072000b410 comp rx=0 tx=0).stop 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.050+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 msgr2=0x7f072c19e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.050+0000 7f073439b640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 0x7f072c19e0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f072402ade0 tx=0x7f0724003ba0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.050+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 shutdown_connections 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.050+0000 7f073439b640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f071807ad80 0x7f071807d240 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.050+0000 7f073439b640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f072c109110 0x7f072c19e0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.051+0000 7f073439b640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f072c108790 0x7f072c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.051+0000 7f073439b640 1 --2- 172.21.15.43:0/467361911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072c102d80 0x7f072c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.051+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 >> 172.21.15.43:0/467361911 conn(0x7f072c0fe7d0 msgr2=0x7f072c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:37.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.051+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 shutdown_connections 2024-09-13T13:29:37.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.051+0000 7f073439b640 1 -- 172.21.15.43:0/467361911 wait complete. 2024-09-13T13:29:37.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.124+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc194108100 con 0x7fc194103b00 2024-09-13T13:29:37.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.124+0000 7fc191ffb640 1 -- 172.21.15.43:0/977954341 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc18406e930 con 0x7fc194103b00 2024-09-13T13:29:37.126 INFO:teuthology.orchestra.run.smithi043.stdout:55834574894 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fc17c07ad80 msgr2=0x7fc17c07d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fc17c07ad80 0x7fc17c07d240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc18802a930 tx=0x7fc1880056d0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc194103b00 msgr2=0x7fc1941987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc194103b00 0x7fc1941987b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc18400dab0 tx=0x7fc18400deb0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 shutdown_connections 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19410d0b0 0x7fc19419e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc194104500 0x7fc194198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fc17c07ad80 0x7fc17c07d240 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 --2- 172.21.15.43:0/977954341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc194103b00 0x7fc1941987b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 >> 172.21.15.43:0/977954341 conn(0x7fc1940fd6d0 msgr2=0x7fc1940fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 shutdown_connections 2024-09-13T13:29:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:37.127+0000 7fc19afe1640 1 -- 172.21.15.43:0/977954341 wait complete. 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: [13/Sep/2024:13:29:36] ENGINE Bus STARTING 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: [13/Sep/2024:13:29:36] ENGINE Serving on http://172.21.15.43:8765 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: [13/Sep/2024:13:29:36] ENGINE Serving on https://172.21.15.43:7150 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: [13/Sep/2024:13:29:36] ENGINE Bus STARTED 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: [13/Sep/2024:13:29:36] ENGINE Error in HTTPServer.serve 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout: Traceback (most recent call last): 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-13T13:29:37.317 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._run(expiration_interval) 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: s = self.context.wrap_socket( 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: return self.sslsocket_class._create( 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:29:37.318 INFO:journalctl@ceph.mon.c.smithi043.stdout: self.do_handshake() 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._sslobj.do_handshake() 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.c.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/467361911' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/977954341' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: [13/Sep/2024:13:29:36] ENGINE Bus STARTING 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-13T13:29:37.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: [13/Sep/2024:13:29:36] ENGINE Serving on http://172.21.15.43:8765 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: [13/Sep/2024:13:29:36] ENGINE Serving on https://172.21.15.43:7150 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: [13/Sep/2024:13:29:36] ENGINE Bus STARTED 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: [13/Sep/2024:13:29:36] ENGINE Error in HTTPServer.serve 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:29:37.320 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/467361911' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T13:29:37.321 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/977954341' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T13:29:37.418 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: [13/Sep/2024:13:29:36] ENGINE Bus STARTING 2024-09-13T13:29:37.418 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:37.418 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-13T13:29:37.418 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: [13/Sep/2024:13:29:36] ENGINE Serving on http://172.21.15.43:8765 2024-09-13T13:29:37.418 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: [13/Sep/2024:13:29:36] ENGINE Serving on https://172.21.15.43:7150 2024-09-13T13:29:37.418 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: [13/Sep/2024:13:29:36] ENGINE Bus STARTED 2024-09-13T13:29:37.418 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: [13/Sep/2024:13:29:36] ENGINE Error in HTTPServer.serve 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: Traceback (most recent call last): 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._connections.run(self.expiration_interval) 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._run(expiration_interval) 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T13:29:37.419 INFO:journalctl@ceph.mon.b.smithi189.stdout: s = self.context.wrap_socket( 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout: return self.sslsocket_class._create( 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout: self.do_handshake() 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._sslobj.do_handshake() 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/467361911' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T13:29:37.420 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/977954341' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T13:29:37.969 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574894 for osd.1 2024-09-13T13:29:37.969 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:38.372 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:38.372 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:38.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411369 for osd.2 2024-09-13T13:29:38.379 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 -- 172.21.15.43:0/3870160926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c1087b0 msgr2=0x7fb61c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3870160926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c1087b0 0x7fb61c108bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb60c009900 tx=0x7fb60c02a410 comp rx=0 tx=0).stop 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 -- 172.21.15.43:0/3870160926 shutdown_connections 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3870160926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c109130 0x7fb61c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3870160926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c1087b0 0x7fb61c108bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3870160926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c102da0 0x7fb61c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 -- 172.21.15.43:0/3870160926 >> 172.21.15.43:0/3870160926 conn(0x7fb61c0fe810 msgr2=0x7fb61c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 -- 172.21.15.43:0/3870160926 shutdown_connections 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.194+0000 7fb623a1e640 1 -- 172.21.15.43:0/3870160926 wait complete. 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 Processor -- start 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 -- start start 2024-09-13T13:29:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c102da0 0x7fb61c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c1087b0 0x7fb61c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c109130 0x7fb61c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1122c0 con 0x7fb61c1087b0 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c112140 con 0x7fb61c109130 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb623a1e640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fb61c112440 con 0x7fb61c102da0 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c102da0 0x7fb61c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb620fb9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c1087b0 0x7fb61c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c102da0 0x7fb61c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48830/0 (socket says 172.21.15.43:48830) 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb620fb9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c1087b0 0x7fb61c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44300/0 (socket says 172.21.15.43:44300) 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 -- 172.21.15.43:0/3037480029 learned_addr learned my addr 172.21.15.43:0/3037480029 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb621fbb640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c109130 0x7fb61c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 -- 172.21.15.43:0/3037480029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c109130 msgr2=0x7fb61c19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:39.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c109130 0x7fb61c19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 -- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c1087b0 msgr2=0x7fb61c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c1087b0 0x7fb61c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb6217ba640 1 -- 172.21.15.43:0/3037480029 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c19e8e0 con 0x7fb61c102da0 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb621fbb640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c109130 0x7fb61c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.195+0000 7fb620fb9640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c1087b0 0x7fb61c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.196+0000 7fb6217ba640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c102da0 0x7fb61c198830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb61800ea60 tx=0x7fb61800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.196+0000 7fb6127fc640 1 -- 172.21.15.43:0/3037480029 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61800cde0 con 0x7fb61c102da0 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.196+0000 7fb6127fc640 1 -- 172.21.15.43:0/3037480029 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb618004470 con 0x7fb61c102da0 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.196+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c19ebd0 con 0x7fb61c102da0 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.196+0000 7fb6127fc640 1 -- 172.21.15.43:0/3037480029 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618010430 con 0x7fb61c102da0 2024-09-13T13:29:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.196+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c1a5310 con 0x7fb61c102da0 2024-09-13T13:29:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.197+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61c103180 con 0x7fb61c102da0 2024-09-13T13:29:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.197+0000 7fb6127fc640 1 -- 172.21.15.43:0/3037480029 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fb618003f60 con 0x7fb61c102da0 2024-09-13T13:29:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.198+0000 7fb6127fc640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb60807ae10 0x7fb60807d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.198+0000 7fb6127fc640 1 -- 172.21.15.43:0/3037480029 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7fb61809e740 con 0x7fb61c102da0 2024-09-13T13:29:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.198+0000 7fb620fb9640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb60807ae10 0x7fb60807d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.198+0000 7fb620fb9640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb60807ae10 0x7fb60807d2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb60c000c00 tx=0x7fb60c0317b0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.203+0000 7fb6127fc640 1 -- 172.21.15.43:0/3037480029 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb6180699f0 con 0x7fb61c102da0 2024-09-13T13:29:39.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.389+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb61c10af10 con 0x7fb61c102da0 2024-09-13T13:29:39.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.390+0000 7fb6127fc640 1 -- 172.21.15.43:0/3037480029 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb61806e8a0 con 0x7fb61c102da0 2024-09-13T13:29:39.391 INFO:teuthology.orchestra.run.smithi043.stdout:210453397515 2024-09-13T13:29:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb60807ae10 msgr2=0x7fb60807d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb60807ae10 0x7fb60807d2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb60c000c00 tx=0x7fb60c0317b0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c102da0 msgr2=0x7fb61c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c102da0 0x7fb61c198830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb61800ea60 tx=0x7fb61800ef20 comp rx=0 tx=0).stop 2024-09-13T13:29:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 shutdown_connections 2024-09-13T13:29:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c109130 0x7fb61c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61c1087b0 0x7fb61c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb60807ae10 0x7fb60807d2d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 --2- 172.21.15.43:0/3037480029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb61c102da0 0x7fb61c198830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 >> 172.21.15.43:0/3037480029 conn(0x7fb61c0fe810 msgr2=0x7fb61c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 shutdown_connections 2024-09-13T13:29:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:39.392+0000 7fb623a1e640 1 -- 172.21.15.43:0/3037480029 wait complete. 2024-09-13T13:29:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:39 smithi189 ceph-mon[32912]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:39 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-13T13:29:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:39 smithi189 ceph-mon[32912]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-13T13:29:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3037480029' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[40255]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[40255]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3037480029' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[30106]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[30106]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-13T13:29:39.976 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3037480029' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T13:29:40.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 -- 172.21.15.43:0/1623830127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c101c60 msgr2=0x7fb06c102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 --2- 172.21.15.43:0/1623830127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c101c60 0x7fb06c102040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009900 tx=0x7fb05c02a3f0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 -- 172.21.15.43:0/1623830127 shutdown_connections 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 --2- 172.21.15.43:0/1623830127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107ff0 0x7fb06c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 --2- 172.21.15.43:0/1623830127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb06c107670 0x7fb06c107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 --2- 172.21.15.43:0/1623830127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c101c60 0x7fb06c102040 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 -- 172.21.15.43:0/1623830127 >> 172.21.15.43:0/1623830127 conn(0x7fb06c0fd6d0 msgr2=0x7fb06c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 -- 172.21.15.43:0/1623830127 shutdown_connections 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.137+0000 7fb072646640 1 -- 172.21.15.43:0/1623830127 wait complete. 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 Processor -- start 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 -- start start 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb06c101c60 0x7fb06c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107670 0x7fb06c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c107ff0 0x7fb06c1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1143d0 con 0x7fb06c107670 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c114250 con 0x7fb06c101c60 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb072646640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fb06c114550 con 0x7fb06c107ff0 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.138+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107670 0x7fb06c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c107ff0 0x7fb06c1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c107ff0 0x7fb06c1a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48874/0 (socket says 172.21.15.43:48874) 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107670 0x7fb06c19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44346/0 (socket says 172.21.15.43:44346) 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 -- 172.21.15.43:0/75206505 learned_addr learned my addr 172.21.15.43:0/75206505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb06bfff640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb06c101c60 0x7fb06c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 -- 172.21.15.43:0/75206505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb06c101c60 msgr2=0x7fb06c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb06c101c60 0x7fb06c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 -- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107670 msgr2=0x7fb06c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107670 0x7fb06c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 -- 172.21.15.43:0/75206505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c1a09e0 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb06bfff640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb06c101c60 0x7fb06c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb06b7fe640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107670 0x7fb06c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb070be3640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c107ff0 0x7fb06c1a0210 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb06400efd0 tx=0x7fb06400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb0697fa640 1 -- 172.21.15.43:0/75206505 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064003b70 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb0697fa640 1 -- 172.21.15.43:0/75206505 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb064003d10 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c1a0cd0 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb0697fa640 1 -- 172.21.15.43:0/75206505 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064010690 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.139+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a7390 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.141+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06c102040 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.141+0000 7fb0697fa640 1 -- 172.21.15.43:0/75206505 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fb064004020 con 0x7fb06c107ff0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.142+0000 7fb0697fa640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb05407ae00 0x7fb05407d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.142+0000 7fb06bfff640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb05407ae00 0x7fb05407d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.142+0000 7fb0697fa640 1 -- 172.21.15.43:0/75206505 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7fb06409e890 con 0x7fb06c107ff0 2024-09-13T13:29:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.143+0000 7fb06bfff640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb05407ae00 0x7fb05407d2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0057c0 tx=0x7fb05c005a20 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.146+0000 7fb0697fa640 1 -- 172.21.15.43:0/75206505 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb064069bc0 con 0x7fb06c107ff0 2024-09-13T13:29:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 -- 172.21.15.43:0/680768822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324102db0 msgr2=0x7f6324103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 --2- 172.21.15.43:0/680768822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324102db0 0x7f6324103190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6314009900 tx=0x7f631402a3f0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 -- 172.21.15.43:0/680768822 shutdown_connections 2024-09-13T13:29:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 --2- 172.21.15.43:0/680768822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6324109140 0x7f632410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 --2- 172.21.15.43:0/680768822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63241087c0 0x7f6324108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 --2- 172.21.15.43:0/680768822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324102db0 0x7f6324103190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 -- 172.21.15.43:0/680768822 >> 172.21.15.43:0/680768822 conn(0x7f63240fe7e0 msgr2=0x7f6324100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 -- 172.21.15.43:0/680768822 shutdown_connections 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.063+0000 7f6328ad4640 1 -- 172.21.15.43:0/680768822 wait complete. 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 Processor -- start 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 -- start start 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6324102db0 0x7f63240754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63241087c0 0x7f63240759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324109140 0x7f6324075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6324112290 con 0x7f6324109140 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6324112110 con 0x7f6324102db0 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6328ad4640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f6324112410 con 0x7f63241087c0 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6321d9b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63241087c0 0x7f63240759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6322d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324109140 0x7f6324075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6321d9b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63241087c0 0x7f63240759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48858/0 (socket says 172.21.15.43:48858) 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6321d9b640 1 -- 172.21.15.43:0/3240915149 learned_addr learned my addr 172.21.15.43:0/3240915149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6322d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324109140 0x7f6324075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44330/0 (socket says 172.21.15.43:44330) 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f632259c640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6324102db0 0x7f63240754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6322d9d640 1 -- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63241087c0 msgr2=0x7f63240759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6322d9d640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63241087c0 0x7f63240759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6322d9d640 1 -- 172.21.15.43:0/3240915149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6324102db0 msgr2=0x7f63240754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6322d9d640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6324102db0 0x7f63240754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f6322d9d640 1 -- 172.21.15.43:0/3240915149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63241a6ba0 con 0x7f6324109140 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.064+0000 7f632259c640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6324102db0 0x7f63240754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.065+0000 7f6322d9d640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324109140 0x7f6324075f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f631c00ec50 tx=0x7f631c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.065+0000 7f63137fe640 1 -- 172.21.15.43:0/3240915149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c003b70 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.065+0000 7f63137fe640 1 -- 172.21.15.43:0/3240915149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f631c003d10 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.065+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63241a6e90 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.065+0000 7f63137fe640 1 -- 172.21.15.43:0/3240915149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c010690 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.065+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63241a72f0 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.066+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6324103190 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.067+0000 7f63137fe640 1 -- 172.21.15.43:0/3240915149 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f631c003fc0 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.067+0000 7f63137fe640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f630c07ae10 0x7f630c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.067+0000 7f63137fe640 1 -- 172.21.15.43:0/3240915149 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7f631c09e9a0 con 0x7f6324109140 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.067+0000 7f632259c640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f630c07ae10 0x7f630c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.068+0000 7f632259c640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f630c07ae10 0x7f630c07d2d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6314000c00 tx=0x7f6314005a20 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.072+0000 7f63137fe640 1 -- 172.21.15.43:0/3240915149 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f631c069c50 con 0x7f6324109140 2024-09-13T13:29:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.263+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f632410a990 con 0x7f6324109140 2024-09-13T13:29:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.264+0000 7f63137fe640 1 -- 172.21.15.43:0/3240915149 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f631c06eb00 con 0x7f6324109140 2024-09-13T13:29:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:184683593744 2024-09-13T13:29:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f630c07ae10 msgr2=0x7f630c07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f630c07ae10 0x7f630c07d2d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6314000c00 tx=0x7f6314005a20 comp rx=0 tx=0).stop 2024-09-13T13:29:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324109140 msgr2=0x7f6324075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324109140 0x7f6324075f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f631c00ec50 tx=0x7f631c00c6a0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 shutdown_connections 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6324109140 0x7f6324075f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f630c07ae10 0x7f630c07d2d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63241087c0 0x7f63240759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 --2- 172.21.15.43:0/3240915149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6324102db0 0x7f63240754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 >> 172.21.15.43:0/3240915149 conn(0x7f63240fe7e0 msgr2=0x7f6324100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 shutdown_connections 2024-09-13T13:29:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.267+0000 7f6328ad4640 1 -- 172.21.15.43:0/3240915149 wait complete. 2024-09-13T13:29:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.343+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb06c1a7650 con 0x7fb06c107ff0 2024-09-13T13:29:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.344+0000 7fb0697fa640 1 -- 172.21.15.43:0/75206505 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb06406c0e0 con 0x7fb06c107ff0 2024-09-13T13:29:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:158913789974 2024-09-13T13:29:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb05407ae00 msgr2=0x7fb05407d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb05407ae00 0x7fb05407d2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0057c0 tx=0x7fb05c005a20 comp rx=0 tx=0).stop 2024-09-13T13:29:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c107ff0 msgr2=0x7fb06c1a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c107ff0 0x7fb06c1a0210 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb06400efd0 tx=0x7fb06400c6a0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 shutdown_connections 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb05407ae00 0x7fb05407d2c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb06c107ff0 0x7fb06c1a0210 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb06c107670 0x7fb06c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 --2- 172.21.15.43:0/75206505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb06c101c60 0x7fb06c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 >> 172.21.15.43:0/75206505 conn(0x7fb06c0fd6d0 msgr2=0x7fb06c1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 shutdown_connections 2024-09-13T13:29:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:40.346+0000 7fb072646640 1 -- 172.21.15.43:0/75206505 wait complete. 2024-09-13T13:29:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3240915149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T13:29:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/75206505' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T13:29:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3240915149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T13:29:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/75206505' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T13:29:41.140 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397514 got 210453397515 for osd.7 2024-09-13T13:29:41.141 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:41.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3240915149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T13:29:41.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/75206505' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T13:29:41.506 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:41.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593743 got 184683593744 for osd.6 2024-09-13T13:29:41.540 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:41.767 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:41 smithi043 ceph-mon[30106]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:41.768 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:41 smithi043 ceph-mon[40255]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:41 smithi189 ceph-mon[32912]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:29:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 -- 172.21.15.43:0/451962301 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 msgr2=0x7fb10c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 --2- 172.21.15.43:0/451962301 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c10f530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ca0 tx=0x7fb10402a8c0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 -- 172.21.15.43:0/451962301 shutdown_connections 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 --2- 172.21.15.43:0/451962301 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c10f530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 --2- 172.21.15.43:0/451962301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 0x7fb10c10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 --2- 172.21.15.43:0/451962301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb10c103b10 0x7fb10c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 -- 172.21.15.43:0/451962301 >> 172.21.15.43:0/451962301 conn(0x7fb10c0fd6e0 msgr2=0x7fb10c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 -- 172.21.15.43:0/451962301 shutdown_connections 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.291+0000 7fb1120a6640 1 -- 172.21.15.43:0/451962301 wait complete. 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 Processor -- start 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 -- start start 2024-09-13T13:29:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb10c103b10 0x7fb10c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 0x7fb10c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c112210 con 0x7fb10c104510 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c112090 con 0x7fb10c103b10 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb1120a6640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c112390 con 0x7fb10c10d0c0 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb10affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 0x7fb10c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.292+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 0x7fb10c198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44370/0 (socket says 172.21.15.43:44370) 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c19e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48890/0 (socket says 172.21.15.43:48890) 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 -- 172.21.15.43:0/972128941 learned_addr learned my addr 172.21.15.43:0/972128941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10b7fe640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb10c103b10 0x7fb10c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 -- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 msgr2=0x7fb10c19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 -- 172.21.15.43:0/972128941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb10c103b10 msgr2=0x7fb10c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb10c103b10 0x7fb10c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 -- 172.21.15.43:0/972128941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c19e8f0 con 0x7fb10c104510 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10bfff640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb10affd640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 0x7fb10c198d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb10000ea60 tx=0x7fb10000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb108ff9640 1 -- 172.21.15.43:0/972128941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10000cd20 con 0x7fb10c104510 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10c19ebe0 con 0x7fb10c104510 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb108ff9640 1 -- 172.21.15.43:0/972128941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb100004470 con 0x7fb10c104510 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb108ff9640 1 -- 172.21.15.43:0/972128941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100003e60 con 0x7fb10c104510 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.293+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10c101a40 con 0x7fb10c104510 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.294+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10c06bb60 con 0x7fb10c104510 2024-09-13T13:29:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.295+0000 7fb108ff9640 1 -- 172.21.15.43:0/972128941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fb100004000 con 0x7fb10c104510 2024-09-13T13:29:42.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.296+0000 7fb108ff9640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb0f407ae10 0x7fb0f407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:42.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.296+0000 7fb108ff9640 1 -- 172.21.15.43:0/972128941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7fb10009e770 con 0x7fb10c104510 2024-09-13T13:29:42.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.296+0000 7fb10b7fe640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb0f407ae10 0x7fb0f407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:42.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.297+0000 7fb10b7fe640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb0f407ae10 0x7fb0f407d2d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009990 tx=0x7fb0fc009110 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:42.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.301+0000 7fb108ff9640 1 -- 172.21.15.43:0/972128941 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb100069980 con 0x7fb10c104510 2024-09-13T13:29:42.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.488+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb10c199970 con 0x7fb10c104510 2024-09-13T13:29:42.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.489+0000 7fb108ff9640 1 -- 172.21.15.43:0/972128941 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb10006e830 con 0x7fb10c104510 2024-09-13T13:29:42.490 INFO:teuthology.orchestra.run.smithi043.stdout:133143986205 2024-09-13T13:29:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb0f407ae10 msgr2=0x7fb0f407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb0f407ae10 0x7fb0f407d2d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009990 tx=0x7fb0fc009110 comp rx=0 tx=0).stop 2024-09-13T13:29:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 msgr2=0x7fb10c198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 0x7fb10c198d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb10000ea60 tx=0x7fb10000ee60 comp rx=0 tx=0).stop 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 shutdown_connections 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb0f407ae10 0x7fb0f407d2d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb10c10d0c0 0x7fb10c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb10c104510 0x7fb10c198d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 --2- 172.21.15.43:0/972128941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb10c103b10 0x7fb10c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 >> 172.21.15.43:0/972128941 conn(0x7fb10c0fd6e0 msgr2=0x7fb10c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 shutdown_connections 2024-09-13T13:29:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.491+0000 7fb1120a6640 1 -- 172.21.15.43:0/972128941 wait complete. 2024-09-13T13:29:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/972128941' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T13:29:42.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/972128941' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T13:29:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/972128941' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T13:29:43.257 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789973 got 158913789974 for osd.5 2024-09-13T13:29:43.257 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:43.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 -- 172.21.15.43:0/147697985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c1045a0 msgr2=0x7fae1c10ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 --2- 172.21.15.43:0/147697985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c1045a0 0x7fae1c10ca90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae10009be0 tx=0x7fae1002a6c0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 -- 172.21.15.43:0/147697985 shutdown_connections 2024-09-13T13:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 --2- 172.21.15.43:0/147697985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c10d0e0 0x7fae1c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 --2- 172.21.15.43:0/147697985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c1045a0 0x7fae1c10ca90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 --2- 172.21.15.43:0/147697985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c103c80 0x7fae1c104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 -- 172.21.15.43:0/147697985 >> 172.21.15.43:0/147697985 conn(0x7fae1c0fd700 msgr2=0x7fae1c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 -- 172.21.15.43:0/147697985 shutdown_connections 2024-09-13T13:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.421+0000 7fae20ce2640 1 -- 172.21.15.43:0/147697985 wait complete. 2024-09-13T13:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 Processor -- start 2024-09-13T13:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 -- start start 2024-09-13T13:29:43.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c103c80 0x7fae1c101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:43.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c1045a0 0x7fae1c101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c10d0e0 0x7fae1c1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c112220 con 0x7fae1c1045a0 2024-09-13T13:29:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1120a0 con 0x7fae1c103c80 2024-09-13T13:29:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae20ce2640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c1123a0 con 0x7fae1c10d0e0 2024-09-13T13:29:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae19d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c1045a0 0x7fae1c101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae19d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c1045a0 0x7fae1c101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44396/0 (socket says 172.21.15.43:44396) 2024-09-13T13:29:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae1ad9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c10d0e0 0x7fae1c1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae19d9b640 1 -- 172.21.15.43:0/4267971212 learned_addr learned my addr 172.21.15.43:0/4267971212 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.422+0000 7fae1a59c640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c103c80 0x7fae1c101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae1ad9d640 1 -- 172.21.15.43:0/4267971212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c103c80 msgr2=0x7fae1c101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae1ad9d640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c103c80 0x7fae1c101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae1ad9d640 1 -- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c1045a0 msgr2=0x7fae1c101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae1ad9d640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c1045a0 0x7fae1c101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae1ad9d640 1 -- 172.21.15.43:0/4267971212 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c1a6bd0 con 0x7fae1c10d0e0 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae19d9b640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c1045a0 0x7fae1c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae1ad9d640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c10d0e0 0x7fae1c1023b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae14005fd0 tx=0x7fae1400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae0b7fe640 1 -- 172.21.15.43:0/4267971212 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1400ccc0 con 0x7fae1c10d0e0 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae0b7fe640 1 -- 172.21.15.43:0/4267971212 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fae14004480 con 0x7fae1c10d0e0 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c1a6ec0 con 0x7fae1c10d0e0 2024-09-13T13:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae0b7fe640 1 -- 172.21.15.43:0/4267971212 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14003d00 con 0x7fae1c10d0e0 2024-09-13T13:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.423+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c1a7370 con 0x7fae1c10d0e0 2024-09-13T13:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.424+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1c104060 con 0x7fae1c10d0e0 2024-09-13T13:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.425+0000 7fae0b7fe640 1 -- 172.21.15.43:0/4267971212 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fae14003ea0 con 0x7fae1c10d0e0 2024-09-13T13:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.425+0000 7fae0b7fe640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fae0407ae10 0x7fae0407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.425+0000 7fae0b7fe640 1 -- 172.21.15.43:0/4267971212 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7fae1409e490 con 0x7fae1c10d0e0 2024-09-13T13:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.425+0000 7fae1a59c640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fae0407ae10 0x7fae0407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.426+0000 7fae1a59c640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fae0407ae10 0x7fae0407d2d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00abe0 tx=0x7fae0c00a2d0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:43.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.430+0000 7fae0b7fe640 1 -- 172.21.15.43:0/4267971212 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fae14009510 con 0x7fae1c10d0e0 2024-09-13T13:29:43.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.617+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fae1c1080c0 con 0x7fae1c10d0e0 2024-09-13T13:29:43.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.617+0000 7fae0b7fe640 1 -- 172.21.15.43:0/4267971212 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae140697c0 con 0x7fae1c10d0e0 2024-09-13T13:29:43.619 INFO:teuthology.orchestra.run.smithi043.stdout:34359738419 2024-09-13T13:29:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fae0407ae10 msgr2=0x7fae0407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fae0407ae10 0x7fae0407d2d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00abe0 tx=0x7fae0c00a2d0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c10d0e0 msgr2=0x7fae1c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c10d0e0 0x7fae1c1023b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae14005fd0 tx=0x7fae1400ec60 comp rx=0 tx=0).stop 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 shutdown_connections 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fae0407ae10 0x7fae0407d2d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae1c10d0e0 0x7fae1c1023b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c1045a0 0x7fae1c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 --2- 172.21.15.43:0/4267971212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c103c80 0x7fae1c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 >> 172.21.15.43:0/4267971212 conn(0x7fae1c0fd700 msgr2=0x7fae1c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 shutdown_connections 2024-09-13T13:29:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.620+0000 7fae20ce2640 1 -- 172.21.15.43:0/4267971212 wait complete. 2024-09-13T13:29:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:43 smithi043 ceph-mon[30106]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-13T13:29:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/903487020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T13:29:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4267971212' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T13:29:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:43 smithi043 ceph-mon[40255]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-13T13:29:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/903487020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T13:29:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4267971212' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T13:29:43.812 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986203 got 133143986205 for osd.4 2024-09-13T13:29:43.812 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:44.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 -- 172.21.15.43:0/2235250591 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8107ff0 msgr2=0x7f11b810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:44.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 --2- 172.21.15.43:0/2235250591 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8107ff0 0x7f11b810e7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009be0 tx=0x7f11b002a6c0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 -- 172.21.15.43:0/2235250591 shutdown_connections 2024-09-13T13:29:44.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 --2- 172.21.15.43:0/2235250591 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8107ff0 0x7f11b810e7f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 --2- 172.21.15.43:0/2235250591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8107670 0x7f11b8107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 --2- 172.21.15.43:0/2235250591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b8101c60 0x7f11b8102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 -- 172.21.15.43:0/2235250591 >> 172.21.15.43:0/2235250591 conn(0x7f11b80fd6d0 msgr2=0x7f11b80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 -- 172.21.15.43:0/2235250591 shutdown_connections 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.960+0000 7f11bea33640 1 -- 172.21.15.43:0/2235250591 wait complete. 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 Processor -- start 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 -- start start 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8101c60 0x7f11b819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8107670 0x7f11b819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b8107ff0 0x7f11b81a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8111140 con 0x7f11b8107ff0 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8110fc0 con 0x7f11b8107670 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bea33640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f11b81112c0 con 0x7f11b8101c60 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11b7fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8101c60 0x7f11b819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11b7fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8101c60 0x7f11b819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48912/0 (socket says 172.21.15.43:48912) 2024-09-13T13:29:44.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11b7fff640 1 -- 172.21.15.43:0/903487020 learned_addr learned my addr 172.21.15.43:0/903487020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.961+0000 7f11bcfd0640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b8107ff0 0x7f11b81a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b77fe640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8107670 0x7f11b819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b7fff640 1 -- 172.21.15.43:0/903487020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8107670 msgr2=0x7f11b819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b7fff640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8107670 0x7f11b819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b7fff640 1 -- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b8107ff0 msgr2=0x7f11b81a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b7fff640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b8107ff0 0x7f11b81a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b7fff640 1 -- 172.21.15.43:0/903487020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b81a0a00 con 0x7f11b8101c60 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b7fff640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8101c60 0x7f11b819a950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e9c0 tx=0x7f11a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b57fa640 1 -- 172.21.15.43:0/903487020 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a800cde0 con 0x7f11b8101c60 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b81a0cf0 con 0x7f11b8101c60 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b57fa640 1 -- 172.21.15.43:0/903487020 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f11a8004470 con 0x7f11b8101c60 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11b57fa640 1 -- 172.21.15.43:0/903487020 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8003e50 con 0x7f11b8101c60 2024-09-13T13:29:44.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.962+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b81a73b0 con 0x7f11b8101c60 2024-09-13T13:29:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.964+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b8102040 con 0x7f11b8101c60 2024-09-13T13:29:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.964+0000 7f11b57fa640 1 -- 172.21.15.43:0/903487020 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f11a80104f0 con 0x7f11b8101c60 2024-09-13T13:29:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.964+0000 7f11b57fa640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f11a007afb0 0x7f11a007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.964+0000 7f11b57fa640 1 -- 172.21.15.43:0/903487020 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7f11a809e700 con 0x7f11b8101c60 2024-09-13T13:29:44.088 INFO:teuthology.orchestra.run.smithi043.stdout:111669149732 2024-09-13T13:29:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.965+0000 7f11b77fe640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f11a007afb0 0x7f11a007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:44.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.965+0000 7f11b77fe640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f11a007afb0 0x7f11a007d470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11b8100be0 tx=0x7f11ac00b390 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:44.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:42.969+0000 7f11b57fa640 1 -- 172.21.15.43:0/903487020 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f11a806a6c0 con 0x7f11b8101c60 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.156+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f11b81a7670 con 0x7f11b8101c60 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.156+0000 7f11b57fa640 1 -- 172.21.15.43:0/903487020 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f11a806f570 con 0x7f11b8101c60 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f11a007afb0 msgr2=0x7f11a007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f11a007afb0 0x7f11a007d470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11b8100be0 tx=0x7f11ac00b390 comp rx=0 tx=0).stop 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8101c60 msgr2=0x7f11b819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8101c60 0x7f11b819a950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e9c0 tx=0x7f11a800ee80 comp rx=0 tx=0).stop 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 shutdown_connections 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b8107ff0 0x7f11b81a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8107670 0x7f11b819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f11a007afb0 0x7f11a007d470 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 --2- 172.21.15.43:0/903487020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f11b8101c60 0x7f11b819a950 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 >> 172.21.15.43:0/903487020 conn(0x7f11b80fd6d0 msgr2=0x7f11b806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 shutdown_connections 2024-09-13T13:29:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:43.158+0000 7f11bea33640 1 -- 172.21.15.43:0/903487020 wait complete. 2024-09-13T13:29:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:43 smithi189 ceph-mon[32912]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-13T13:29:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/903487020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T13:29:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4267971212' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T13:29:44.767 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149732 for osd.3 2024-09-13T13:29:44.767 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:45.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738419 for osd.0 2024-09-13T13:29:45.144 DEBUG:teuthology.parallel:result is None 2024-09-13T13:29:45.144 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-13T13:29:45.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T13:29:45.436 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:45.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:29:45.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:45 smithi043 ceph-mon[30106]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-13T13:29:45.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:45 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:45.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:45 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:45.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:45 smithi043 ceph-mon[40255]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-13T13:29:45.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:45 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:45.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:45 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:45.849 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:29:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:29:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3126339628 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 msgr2=0x7f85a0107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3126339628 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 0x7f85a0107ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8588009900 tx=0x7f858802a410 comp rx=0 tx=0).stop 2024-09-13T13:29:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3126339628 shutdown_connections 2024-09-13T13:29:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3126339628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 0x7f85a010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3126339628 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 0x7f85a0107ab0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3126339628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85a0101c60 0x7f85a0102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3126339628 >> 172.21.15.43:0/3126339628 conn(0x7f85a00fd6d0 msgr2=0x7f85a00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3126339628 shutdown_connections 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3126339628 wait complete. 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.847+0000 7f85a5dbc640 1 Processor -- start 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f85a5dbc640 1 -- start start 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f85a5dbc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85a0101c60 0x7f85a019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f85a5dbc640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 0x7f85a019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f85a5dbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 0x7f85a01a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f85a5dbc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01111d0 con 0x7f85a0107ff0 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f85a5dbc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0111050 con 0x7f85a0101c60 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f85a5dbc640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f85a0111350 con 0x7f85a0107670 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f859ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 0x7f85a01a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f859ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 0x7f85a01a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44414/0 (socket says 172.21.15.43:44414) 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f859ffff640 1 -- 172.21.15.43:0/3520207510 learned_addr learned my addr 172.21.15.43:0/3520207510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f859f7fe640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85a0101c60 0x7f85a019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.848+0000 7f8596ffd640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 0x7f85a019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859ffff640 1 -- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 msgr2=0x7f85a019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859ffff640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 0x7f85a019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859ffff640 1 -- 172.21.15.43:0/3520207510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85a0101c60 msgr2=0x7f85a019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859ffff640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85a0101c60 0x7f85a019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859ffff640 1 -- 172.21.15.43:0/3520207510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a01a4e70 con 0x7f85a0107ff0 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859ffff640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 0x7f85a01a46a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f859800eb10 tx=0x7f859800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859d7fa640 1 -- 172.21.15.43:0/3520207510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598004b20 con 0x7f85a0107ff0 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a01a5160 con 0x7f85a0107ff0 2024-09-13T13:29:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859d7fa640 1 -- 172.21.15.43:0/3520207510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f8598004cc0 con 0x7f85a0107ff0 2024-09-13T13:29:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f859d7fa640 1 -- 172.21.15.43:0/3520207510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598010430 con 0x7f85a0107ff0 2024-09-13T13:29:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.849+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01ab820 con 0x7f85a0107ff0 2024-09-13T13:29:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.850+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a0102040 con 0x7f85a0107ff0 2024-09-13T13:29:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.851+0000 7f859d7fa640 1 -- 172.21.15.43:0/3520207510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f8598010610 con 0x7f85a0107ff0 2024-09-13T13:29:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.852+0000 7f859d7fa640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f858c07ae10 0x7f858c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.852+0000 7f859f7fe640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f858c07ae10 0x7f858c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.852+0000 7f859d7fa640 1 -- 172.21.15.43:0/3520207510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7f859809e960 con 0x7f85a0107ff0 2024-09-13T13:29:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.852+0000 7f859f7fe640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f858c07ae10 0x7f858c07d2d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f859000bbe0 tx=0x7f859000b390 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:45.856+0000 7f859d7fa640 1 -- 172.21.15.43:0/3520207510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f8598069c10 con 0x7f85a0107ff0 2024-09-13T13:29:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.044+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 --> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85a0103c20 con 0x7f858c07ae10 2024-09-13T13:29:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.051+0000 7f859d7fa640 1 -- 172.21.15.43:0/3520207510 <== mgr.24458 v2:172.21.15.43:6800/3978596566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346450 (secure 0 0 0) 0x7f85a0103c20 con 0x7f858c07ae10 2024-09-13T13:29:46.056 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:29:46.058 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-13T13:29:46.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f858c07ae10 msgr2=0x7f858c07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:46.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f858c07ae10 0x7f858c07d2d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f859000bbe0 tx=0x7f859000b390 comp rx=0 tx=0).stop 2024-09-13T13:29:46.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 msgr2=0x7f85a01a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 0x7f85a01a46a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f859800eb10 tx=0x7f859800c2d0 comp rx=0 tx=0).stop 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 shutdown_connections 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0107ff0 0x7f85a01a46a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f858c07ae10 0x7f858c07d2d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f85a0107670 0x7f85a019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 --2- 172.21.15.43:0/3520207510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85a0101c60 0x7f85a019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 >> 172.21.15.43:0/3520207510 conn(0x7f85a00fd6d0 msgr2=0x7f85a00ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 shutdown_connections 2024-09-13T13:29:46.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:46.055+0000 7f85a5dbc640 1 -- 172.21.15.43:0/3520207510 wait complete. 2024-09-13T13:29:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:45 smithi189 ceph-mon[32912]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-13T13:29:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:45 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:45 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:46.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:46.993 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-13T13:29:44.167666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":880,"num_read_kb":737,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":235960,"kb_used_data":7260,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725256,"statfs":{"total":767960285184,"available":767718662144,"internally_reserved":0,"allocated":7434240,"data_stored":4068817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":181,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.996790"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837131+0000","last_change":"2024-09-13T13:29:02.691119+0000","last_active":"2024-09-13T13:29:34.837131+0000","last_peered":"2024-09-13T13:29:34.837131+0000","last_clean":"2024-09-13T13:29:34.837131+0000","last_became_active":"2024-09-13T13:29:02.690686+0000","last_became_peered":"2024-09-13T13:29:02.690686+0000","last_unstale":"2024-09-13T13:29:34.837131+0000","last_undegraded":"2024-09-13T13:29:34.837131+0000","last_fullsized":"2024-09-13T13:29:34.837131+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:44:55.882708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836461+0000","last_change":"2024-09-13T13:29:04.698113+0000","last_active":"2024-09-13T13:29:34.836461+0000","last_peered":"2024-09-13T13:29:34.836461+0000","last_clean":"2024-09-13T13:29:34.836461+0000","last_became_active":"2024-09-13T13:29:04.696445+0000","last_became_peered":"2024-09-13T13:29:04.696445+0000","last_unstale":"2024-09-13T13:29:34.836461+0000","last_undegraded":"2024-09-13T13:29:34.836461+0000","last_fullsized":"2024-09-13T13:29:34.836461+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:44.558237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837363+0000","last_change":"2024-09-13T13:29:06.696895+0000","last_active":"2024-09-13T13:29:34.837363+0000","last_peered":"2024-09-13T13:29:34.837363+0000","last_clean":"2024-09-13T13:29:34.837363+0000","last_became_active":"2024-09-13T13:29:06.696575+0000","last_became_peered":"2024-09-13T13:29:06.696575+0000","last_unstale":"2024-09-13T13:29:34.837363+0000","last_undegraded":"2024-09-13T13:29:34.837363+0000","last_fullsized":"2024-09-13T13:29:34.837363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:01.389734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836485+0000","last_change":"2024-09-13T13:29:08.710602+0000","last_active":"2024-09-13T13:29:34.836485+0000","last_peered":"2024-09-13T13:29:34.836485+0000","last_clean":"2024-09-13T13:29:34.836485+0000","last_became_active":"2024-09-13T13:29:08.710327+0000","last_became_peered":"2024-09-13T13:29:08.710327+0000","last_unstale":"2024-09-13T13:29:34.836485+0000","last_undegraded":"2024-09-13T13:29:34.836485+0000","last_fullsized":"2024-09-13T13:29:34.836485+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:03.826279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153202+0000","last_change":"2024-09-13T13:29:08.713800+0000","last_active":"2024-09-13T13:29:34.153202+0000","last_peered":"2024-09-13T13:29:34.153202+0000","last_clean":"2024-09-13T13:29:34.153202+0000","last_became_active":"2024-09-13T13:29:08.713188+0000","last_became_peered":"2024-09-13T13:29:08.713188+0000","last_unstale":"2024-09-13T13:29:34.153202+0000","last_undegraded":"2024-09-13T13:29:34.153202+0000","last_fullsized":"2024-09-13T13:29:34.153202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:38.132777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155384+0000","last_change":"2024-09-13T13:29:02.676910+0000","last_active":"2024-09-13T13:29:34.155384+0000","last_peered":"2024-09-13T13:29:34.155384+0000","last_clean":"2024-09-13T13:29:34.155384+0000","last_became_active":"2024-09-13T13:29:02.676635+0000","last_became_peered":"2024-09-13T13:29:02.676635+0000","last_unstale":"2024-09-13T13:29:34.155384+0000","last_undegraded":"2024-09-13T13:29:34.155384+0000","last_fullsized":"2024-09-13T13:29:34.155384+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:32.568003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153030+0000","last_change":"2024-09-13T13:29:04.690627+0000","last_active":"2024-09-13T13:29:34.153030+0000","last_peered":"2024-09-13T13:29:34.153030+0000","last_clean":"2024-09-13T13:29:34.153030+0000","last_became_active":"2024-09-13T13:29:04.690322+0000","last_became_peered":"2024-09-13T13:29:04.690322+0000","last_unstale":"2024-09-13T13:29:34.153030+0000","last_undegraded":"2024-09-13T13:29:34.153030+0000","last_fullsized":"2024-09-13T13:29:34.153030+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:17:24.264025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836353+0000","last_change":"2024-09-13T13:29:06.697516+0000","last_active":"2024-09-13T13:29:34.836353+0000","last_peered":"2024-09-13T13:29:34.836353+0000","last_clean":"2024-09-13T13:29:34.836353+0000","last_became_active":"2024-09-13T13:29:06.696915+0000","last_became_peered":"2024-09-13T13:29:06.696915+0000","last_unstale":"2024-09-13T13:29:34.836353+0000","last_undegraded":"2024-09-13T13:29:34.836353+0000","last_fullsized":"2024-09-13T13:29:34.836353+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:24.892630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644921+0000","last_change":"2024-09-13T13:29:02.682927+0000","last_active":"2024-09-13T13:29:34.644921+0000","last_peered":"2024-09-13T13:29:34.644921+0000","last_clean":"2024-09-13T13:29:34.644921+0000","last_became_active":"2024-09-13T13:29:02.682512+0000","last_became_peered":"2024-09-13T13:29:02.682512+0000","last_unstale":"2024-09-13T13:29:34.644921+0000","last_undegraded":"2024-09-13T13:29:34.644921+0000","last_fullsized":"2024-09-13T13:29:34.644921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:21.323157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836850+0000","last_change":"2024-09-13T13:29:04.687830+0000","last_active":"2024-09-13T13:29:34.836850+0000","last_peered":"2024-09-13T13:29:34.836850+0000","last_clean":"2024-09-13T13:29:34.836850+0000","last_became_active":"2024-09-13T13:29:04.687330+0000","last_became_peered":"2024-09-13T13:29:04.687330+0000","last_unstale":"2024-09-13T13:29:34.836850+0000","last_undegraded":"2024-09-13T13:29:34.836850+0000","last_fullsized":"2024-09-13T13:29:34.836850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:02.875214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644409+0000","last_change":"2024-09-13T13:29:06.702347+0000","last_active":"2024-09-13T13:29:34.644409+0000","last_peered":"2024-09-13T13:29:34.644409+0000","last_clean":"2024-09-13T13:29:34.644409+0000","last_became_active":"2024-09-13T13:29:06.701982+0000","last_became_peered":"2024-09-13T13:29:06.701982+0000","last_unstale":"2024-09-13T13:29:34.644409+0000","last_undegraded":"2024-09-13T13:29:34.644409+0000","last_fullsized":"2024-09-13T13:29:34.644409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:57:11.424454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838568+0000","last_change":"2024-09-13T13:29:08.710877+0000","last_active":"2024-09-13T13:29:34.838568+0000","last_peered":"2024-09-13T13:29:34.838568+0000","last_clean":"2024-09-13T13:29:34.838568+0000","last_became_active":"2024-09-13T13:29:08.710555+0000","last_became_peered":"2024-09-13T13:29:08.710555+0000","last_unstale":"2024-09-13T13:29:34.838568+0000","last_undegraded":"2024-09-13T13:29:34.838568+0000","last_fullsized":"2024-09-13T13:29:34.838568+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:10.659433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644577+0000","last_change":"2024-09-13T13:29:02.683577+0000","last_active":"2024-09-13T13:29:34.644577+0000","last_peered":"2024-09-13T13:29:34.644577+0000","last_clean":"2024-09-13T13:29:34.644577+0000","last_became_active":"2024-09-13T13:29:02.682486+0000","last_became_peered":"2024-09-13T13:29:02.682486+0000","last_unstale":"2024-09-13T13:29:34.644577+0000","last_undegraded":"2024-09-13T13:29:34.644577+0000","last_fullsized":"2024-09-13T13:29:34.644577+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:05.767012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"61'5","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836611+0000","last_change":"2024-09-13T13:29:04.697476+0000","last_active":"2024-09-13T13:29:34.836611+0000","last_peered":"2024-09-13T13:29:34.836611+0000","last_clean":"2024-09-13T13:29:34.836611+0000","last_became_active":"2024-09-13T13:29:04.697175+0000","last_became_peered":"2024-09-13T13:29:04.697175+0000","last_unstale":"2024-09-13T13:29:34.836611+0000","last_undegraded":"2024-09-13T13:29:34.836611+0000","last_fullsized":"2024-09-13T13:29:34.836611+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:43:44.234827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153938+0000","last_change":"2024-09-13T13:29:06.696705+0000","last_active":"2024-09-13T13:29:34.153938+0000","last_peered":"2024-09-13T13:29:34.153938+0000","last_clean":"2024-09-13T13:29:34.153938+0000","last_became_active":"2024-09-13T13:29:06.696422+0000","last_became_peered":"2024-09-13T13:29:06.696422+0000","last_unstale":"2024-09-13T13:29:34.153938+0000","last_undegraded":"2024-09-13T13:29:34.153938+0000","last_fullsized":"2024-09-13T13:29:34.153938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:41.057664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644302+0000","last_change":"2024-09-13T13:29:08.715361+0000","last_active":"2024-09-13T13:29:34.644302+0000","last_peered":"2024-09-13T13:29:34.644302+0000","last_clean":"2024-09-13T13:29:34.644302+0000","last_became_active":"2024-09-13T13:29:08.715023+0000","last_became_peered":"2024-09-13T13:29:08.715023+0000","last_unstale":"2024-09-13T13:29:34.644302+0000","last_undegraded":"2024-09-13T13:29:34.644302+0000","last_fullsized":"2024-09-13T13:29:34.644302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:28:15.702074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836892+0000","last_change":"2024-09-13T13:29:08.704334+0000","last_active":"2024-09-13T13:29:34.836892+0000","last_peered":"2024-09-13T13:29:34.836892+0000","last_clean":"2024-09-13T13:29:34.836892+0000","last_became_active":"2024-09-13T13:29:08.704066+0000","last_became_peered":"2024-09-13T13:29:08.704066+0000","last_unstale":"2024-09-13T13:29:34.836892+0000","last_undegraded":"2024-09-13T13:29:34.836892+0000","last_fullsized":"2024-09-13T13:29:34.836892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:06.053516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.840017+0000","last_change":"2024-09-13T13:29:02.689878+0000","last_active":"2024-09-13T13:29:34.840017+0000","last_peered":"2024-09-13T13:29:34.840017+0000","last_clean":"2024-09-13T13:29:34.840017+0000","last_became_active":"2024-09-13T13:29:02.689376+0000","last_became_peered":"2024-09-13T13:29:02.689376+0000","last_unstale":"2024-09-13T13:29:34.840017+0000","last_undegraded":"2024-09-13T13:29:34.840017+0000","last_fullsized":"2024-09-13T13:29:34.840017+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:34.021660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158393+0000","last_change":"2024-09-13T13:29:04.694389+0000","last_active":"2024-09-13T13:29:34.158393+0000","last_peered":"2024-09-13T13:29:34.158393+0000","last_clean":"2024-09-13T13:29:34.158393+0000","last_became_active":"2024-09-13T13:29:04.694036+0000","last_became_peered":"2024-09-13T13:29:04.694036+0000","last_unstale":"2024-09-13T13:29:34.158393+0000","last_undegraded":"2024-09-13T13:29:34.158393+0000","last_fullsized":"2024-09-13T13:29:34.158393+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:09.377832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841596+0000","last_change":"2024-09-13T13:29:06.705745+0000","last_active":"2024-09-13T13:29:34.841596+0000","last_peered":"2024-09-13T13:29:34.841596+0000","last_clean":"2024-09-13T13:29:34.841596+0000","last_became_active":"2024-09-13T13:29:06.705022+0000","last_became_peered":"2024-09-13T13:29:06.705022+0000","last_unstale":"2024-09-13T13:29:34.841596+0000","last_undegraded":"2024-09-13T13:29:34.841596+0000","last_fullsized":"2024-09-13T13:29:34.841596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:27.783699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155427+0000","last_change":"2024-09-13T13:29:08.711913+0000","last_active":"2024-09-13T13:29:34.155427+0000","last_peered":"2024-09-13T13:29:34.155427+0000","last_clean":"2024-09-13T13:29:34.155427+0000","last_became_active":"2024-09-13T13:29:08.711608+0000","last_became_peered":"2024-09-13T13:29:08.711608+0000","last_unstale":"2024-09-13T13:29:34.155427+0000","last_undegraded":"2024-09-13T13:29:34.155427+0000","last_fullsized":"2024-09-13T13:29:34.155427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:04.109831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.835972+0000","last_change":"2024-09-13T13:29:02.677534+0000","last_active":"2024-09-13T13:29:34.835972+0000","last_peered":"2024-09-13T13:29:34.835972+0000","last_clean":"2024-09-13T13:29:34.835972+0000","last_became_active":"2024-09-13T13:29:02.677185+0000","last_became_peered":"2024-09-13T13:29:02.677185+0000","last_unstale":"2024-09-13T13:29:34.835972+0000","last_undegraded":"2024-09-13T13:29:34.835972+0000","last_fullsized":"2024-09-13T13:29:34.835972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:59.858960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155573+0000","last_change":"2024-09-13T13:29:04.698386+0000","last_active":"2024-09-13T13:29:34.155573+0000","last_peered":"2024-09-13T13:29:34.155573+0000","last_clean":"2024-09-13T13:29:34.155573+0000","last_became_active":"2024-09-13T13:29:04.698086+0000","last_became_peered":"2024-09-13T13:29:04.698086+0000","last_unstale":"2024-09-13T13:29:34.155573+0000","last_undegraded":"2024-09-13T13:29:34.155573+0000","last_fullsized":"2024-09-13T13:29:34.155573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:33.433059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836856+0000","last_change":"2024-09-13T13:29:06.704601+0000","last_active":"2024-09-13T13:29:34.836856+0000","last_peered":"2024-09-13T13:29:34.836856+0000","last_clean":"2024-09-13T13:29:34.836856+0000","last_became_active":"2024-09-13T13:29:06.704322+0000","last_became_peered":"2024-09-13T13:29:06.704322+0000","last_unstale":"2024-09-13T13:29:34.836856+0000","last_undegraded":"2024-09-13T13:29:34.836856+0000","last_fullsized":"2024-09-13T13:29:34.836856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:39.699228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154167+0000","last_change":"2024-09-13T13:29:08.711812+0000","last_active":"2024-09-13T13:29:34.154167+0000","last_peered":"2024-09-13T13:29:34.154167+0000","last_clean":"2024-09-13T13:29:34.154167+0000","last_became_active":"2024-09-13T13:29:08.711491+0000","last_became_peered":"2024-09-13T13:29:08.711491+0000","last_unstale":"2024-09-13T13:29:34.154167+0000","last_undegraded":"2024-09-13T13:29:34.154167+0000","last_fullsized":"2024-09-13T13:29:34.154167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.677336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836598+0000","last_change":"2024-09-13T13:29:02.681192+0000","last_active":"2024-09-13T13:29:34.836598+0000","last_peered":"2024-09-13T13:29:34.836598+0000","last_clean":"2024-09-13T13:29:34.836598+0000","last_became_active":"2024-09-13T13:29:02.680881+0000","last_became_peered":"2024-09-13T13:29:02.680881+0000","last_unstale":"2024-09-13T13:29:34.836598+0000","last_undegraded":"2024-09-13T13:29:34.836598+0000","last_fullsized":"2024-09-13T13:29:34.836598+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:14:08.864821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839546+0000","last_change":"2024-09-13T13:29:04.800696+0000","last_active":"2024-09-13T13:29:34.839546+0000","last_peered":"2024-09-13T13:29:34.839546+0000","last_clean":"2024-09-13T13:29:34.839546+0000","last_became_active":"2024-09-13T13:29:04.800292+0000","last_became_peered":"2024-09-13T13:29:04.800292+0000","last_unstale":"2024-09-13T13:29:34.839546+0000","last_undegraded":"2024-09-13T13:29:34.839546+0000","last_fullsized":"2024-09-13T13:29:34.839546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:43:31.695333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"61'8","reported_seq":44,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493564+0000","last_change":"2024-09-13T13:29:06.701036+0000","last_active":"2024-09-13T13:29:34.493564+0000","last_peered":"2024-09-13T13:29:34.493564+0000","last_clean":"2024-09-13T13:29:34.493564+0000","last_became_active":"2024-09-13T13:29:06.700680+0000","last_became_peered":"2024-09-13T13:29:06.700680+0000","last_unstale":"2024-09-13T13:29:34.493564+0000","last_undegraded":"2024-09-13T13:29:34.493564+0000","last_fullsized":"2024-09-13T13:29:34.493564+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:54.108253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841812+0000","last_change":"2024-09-13T13:29:04.696291+0000","last_active":"2024-09-13T13:29:34.841812+0000","last_peered":"2024-09-13T13:29:34.841812+0000","last_clean":"2024-09-13T13:29:34.841812+0000","last_became_active":"2024-09-13T13:29:04.695972+0000","last_became_peered":"2024-09-13T13:29:04.695972+0000","last_unstale":"2024-09-13T13:29:34.841812+0000","last_undegraded":"2024-09-13T13:29:34.841812+0000","last_fullsized":"2024-09-13T13:29:34.841812+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:41.599634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153155+0000","last_change":"2024-09-13T13:29:02.679531+0000","last_active":"2024-09-13T13:29:34.153155+0000","last_peered":"2024-09-13T13:29:34.153155+0000","last_clean":"2024-09-13T13:29:34.153155+0000","last_became_active":"2024-09-13T13:29:02.679253+0000","last_became_peered":"2024-09-13T13:29:02.679253+0000","last_unstale":"2024-09-13T13:29:34.153155+0000","last_undegraded":"2024-09-13T13:29:34.153155+0000","last_fullsized":"2024-09-13T13:29:34.153155+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.644719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"61'8","reported_seq":38,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837051+0000","last_change":"2024-09-13T13:29:06.701808+0000","last_active":"2024-09-13T13:29:34.837051+0000","last_peered":"2024-09-13T13:29:34.837051+0000","last_clean":"2024-09-13T13:29:34.837051+0000","last_became_active":"2024-09-13T13:29:06.701539+0000","last_became_peered":"2024-09-13T13:29:06.701539+0000","last_unstale":"2024-09-13T13:29:34.837051+0000","last_undegraded":"2024-09-13T13:29:34.837051+0000","last_fullsized":"2024-09-13T13:29:34.837051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:48:55.894905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644565+0000","last_change":"2024-09-13T13:29:08.710820+0000","last_active":"2024-09-13T13:29:34.644565+0000","last_peered":"2024-09-13T13:29:34.644565+0000","last_clean":"2024-09-13T13:29:34.644565+0000","last_became_active":"2024-09-13T13:29:08.710490+0000","last_became_peered":"2024-09-13T13:29:08.710490+0000","last_unstale":"2024-09-13T13:29:34.644565+0000","last_undegraded":"2024-09-13T13:29:34.644565+0000","last_fullsized":"2024-09-13T13:29:34.644565+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:52.100901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"61'18","reported_seq":58,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152862+0000","last_change":"2024-09-13T13:29:04.801791+0000","last_active":"2024-09-13T13:29:34.152862+0000","last_peered":"2024-09-13T13:29:34.152862+0000","last_clean":"2024-09-13T13:29:34.152862+0000","last_became_active":"2024-09-13T13:29:04.801414+0000","last_became_peered":"2024-09-13T13:29:04.801414+0000","last_unstale":"2024-09-13T13:29:34.152862+0000","last_undegraded":"2024-09-13T13:29:34.152862+0000","last_fullsized":"2024-09-13T13:29:34.152862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:09.169611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155649+0000","last_change":"2024-09-13T13:29:02.690016+0000","last_active":"2024-09-13T13:29:34.155649+0000","last_peered":"2024-09-13T13:29:34.155649+0000","last_clean":"2024-09-13T13:29:34.155649+0000","last_became_active":"2024-09-13T13:29:02.689755+0000","last_became_peered":"2024-09-13T13:29:02.689755+0000","last_unstale":"2024-09-13T13:29:34.155649+0000","last_undegraded":"2024-09-13T13:29:34.155649+0000","last_fullsized":"2024-09-13T13:29:34.155649+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:40.036114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836481+0000","last_change":"2024-09-13T13:29:06.698052+0000","last_active":"2024-09-13T13:29:34.836481+0000","last_peered":"2024-09-13T13:29:34.836481+0000","last_clean":"2024-09-13T13:29:34.836481+0000","last_became_active":"2024-09-13T13:29:06.697745+0000","last_became_peered":"2024-09-13T13:29:06.697745+0000","last_unstale":"2024-09-13T13:29:34.836481+0000","last_undegraded":"2024-09-13T13:29:34.836481+0000","last_fullsized":"2024-09-13T13:29:34.836481+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:49.392569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836639+0000","last_change":"2024-09-13T13:29:08.710852+0000","last_active":"2024-09-13T13:29:34.836639+0000","last_peered":"2024-09-13T13:29:34.836639+0000","last_clean":"2024-09-13T13:29:34.836639+0000","last_became_active":"2024-09-13T13:29:08.709926+0000","last_became_peered":"2024-09-13T13:29:08.709926+0000","last_unstale":"2024-09-13T13:29:34.836639+0000","last_undegraded":"2024-09-13T13:29:34.836639+0000","last_fullsized":"2024-09-13T13:29:34.836639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:56.759617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"61'14","reported_seq":47,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836853+0000","last_change":"2024-09-13T13:29:04.686558+0000","last_active":"2024-09-13T13:29:34.836853+0000","last_peered":"2024-09-13T13:29:34.836853+0000","last_clean":"2024-09-13T13:29:34.836853+0000","last_became_active":"2024-09-13T13:29:04.686224+0000","last_became_peered":"2024-09-13T13:29:04.686224+0000","last_unstale":"2024-09-13T13:29:34.836853+0000","last_undegraded":"2024-09-13T13:29:34.836853+0000","last_fullsized":"2024-09-13T13:29:34.836853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:36:08.808981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839906+0000","last_change":"2024-09-13T13:29:02.692710+0000","last_active":"2024-09-13T13:29:34.839906+0000","last_peered":"2024-09-13T13:29:34.839906+0000","last_clean":"2024-09-13T13:29:34.839906+0000","last_became_active":"2024-09-13T13:29:02.692356+0000","last_became_peered":"2024-09-13T13:29:02.692356+0000","last_unstale":"2024-09-13T13:29:34.839906+0000","last_undegraded":"2024-09-13T13:29:34.839906+0000","last_fullsized":"2024-09-13T13:29:34.839906+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:11.431540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"61'8","reported_seq":38,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152685+0000","last_change":"2024-09-13T13:29:06.696788+0000","last_active":"2024-09-13T13:29:34.152685+0000","last_peered":"2024-09-13T13:29:34.152685+0000","last_clean":"2024-09-13T13:29:34.152685+0000","last_became_active":"2024-09-13T13:29:06.696486+0000","last_became_peered":"2024-09-13T13:29:06.696486+0000","last_unstale":"2024-09-13T13:29:34.152685+0000","last_undegraded":"2024-09-13T13:29:34.152685+0000","last_fullsized":"2024-09-13T13:29:34.152685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:10.837492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153817+0000","last_change":"2024-09-13T13:29:08.708590+0000","last_active":"2024-09-13T13:29:34.153817+0000","last_peered":"2024-09-13T13:29:34.153817+0000","last_clean":"2024-09-13T13:29:34.153817+0000","last_became_active":"2024-09-13T13:29:08.708295+0000","last_became_peered":"2024-09-13T13:29:08.708295+0000","last_unstale":"2024-09-13T13:29:34.153817+0000","last_undegraded":"2024-09-13T13:29:34.153817+0000","last_fullsized":"2024-09-13T13:29:34.153817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:11.005570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841528+0000","last_change":"2024-09-13T13:29:04.696010+0000","last_active":"2024-09-13T13:29:34.841528+0000","last_peered":"2024-09-13T13:29:34.841528+0000","last_clean":"2024-09-13T13:29:34.841528+0000","last_became_active":"2024-09-13T13:29:04.695623+0000","last_became_peered":"2024-09-13T13:29:04.695623+0000","last_unstale":"2024-09-13T13:29:34.841528+0000","last_undegraded":"2024-09-13T13:29:34.841528+0000","last_fullsized":"2024-09-13T13:29:34.841528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:16.775741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644975+0000","last_change":"2024-09-13T13:29:02.682664+0000","last_active":"2024-09-13T13:29:34.644975+0000","last_peered":"2024-09-13T13:29:34.644975+0000","last_clean":"2024-09-13T13:29:34.644975+0000","last_became_active":"2024-09-13T13:29:02.682242+0000","last_became_peered":"2024-09-13T13:29:02.682242+0000","last_unstale":"2024-09-13T13:29:34.644975+0000","last_undegraded":"2024-09-13T13:29:34.644975+0000","last_fullsized":"2024-09-13T13:29:34.644975+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:07.397695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"61'8","reported_seq":38,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.840376+0000","last_change":"2024-09-13T13:29:06.706857+0000","last_active":"2024-09-13T13:29:34.840376+0000","last_peered":"2024-09-13T13:29:34.840376+0000","last_clean":"2024-09-13T13:29:34.840376+0000","last_became_active":"2024-09-13T13:29:06.706578+0000","last_became_peered":"2024-09-13T13:29:06.706578+0000","last_unstale":"2024-09-13T13:29:34.840376+0000","last_undegraded":"2024-09-13T13:29:34.840376+0000","last_fullsized":"2024-09-13T13:29:34.840376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:47:33.502240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838762+0000","last_change":"2024-09-13T13:29:08.710401+0000","last_active":"2024-09-13T13:29:34.838762+0000","last_peered":"2024-09-13T13:29:34.838762+0000","last_clean":"2024-09-13T13:29:34.838762+0000","last_became_active":"2024-09-13T13:29:08.709855+0000","last_became_peered":"2024-09-13T13:29:08.709855+0000","last_unstale":"2024-09-13T13:29:34.838762+0000","last_undegraded":"2024-09-13T13:29:34.838762+0000","last_fullsized":"2024-09-13T13:29:34.838762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:10.255479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"61'19","reported_seq":62,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.840540+0000","last_change":"2024-09-13T13:29:04.800191+0000","last_active":"2024-09-13T13:29:34.840540+0000","last_peered":"2024-09-13T13:29:34.840540+0000","last_clean":"2024-09-13T13:29:34.840540+0000","last_became_active":"2024-09-13T13:29:04.799486+0000","last_became_peered":"2024-09-13T13:29:04.799486+0000","last_unstale":"2024-09-13T13:29:34.840540+0000","last_undegraded":"2024-09-13T13:29:34.840540+0000","last_fullsized":"2024-09-13T13:29:34.840540+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:34.012633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836657+0000","last_change":"2024-09-13T13:29:02.683196+0000","last_active":"2024-09-13T13:29:34.836657+0000","last_peered":"2024-09-13T13:29:34.836657+0000","last_clean":"2024-09-13T13:29:34.836657+0000","last_became_active":"2024-09-13T13:29:02.682794+0000","last_became_peered":"2024-09-13T13:29:02.682794+0000","last_unstale":"2024-09-13T13:29:34.836657+0000","last_undegraded":"2024-09-13T13:29:34.836657+0000","last_fullsized":"2024-09-13T13:29:34.836657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:19:49.873368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493107+0000","last_change":"2024-09-13T13:29:06.701848+0000","last_active":"2024-09-13T13:29:34.493107+0000","last_peered":"2024-09-13T13:29:34.493107+0000","last_clean":"2024-09-13T13:29:34.493107+0000","last_became_active":"2024-09-13T13:29:06.701560+0000","last_became_peered":"2024-09-13T13:29:06.701560+0000","last_unstale":"2024-09-13T13:29:34.493107+0000","last_undegraded":"2024-09-13T13:29:34.493107+0000","last_fullsized":"2024-09-13T13:29:34.493107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:33.137042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837360+0000","last_change":"2024-09-13T13:29:08.709068+0000","last_active":"2024-09-13T13:29:34.837360+0000","last_peered":"2024-09-13T13:29:34.837360+0000","last_clean":"2024-09-13T13:29:34.837360+0000","last_became_active":"2024-09-13T13:29:08.708740+0000","last_became_peered":"2024-09-13T13:29:08.708740+0000","last_unstale":"2024-09-13T13:29:34.837360+0000","last_undegraded":"2024-09-13T13:29:34.837360+0000","last_fullsized":"2024-09-13T13:29:34.837360+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:47.118671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"61'28","reported_seq":82,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841657+0000","last_change":"2024-09-13T13:29:04.696278+0000","last_active":"2024-09-13T13:29:34.841657+0000","last_peered":"2024-09-13T13:29:34.841657+0000","last_clean":"2024-09-13T13:29:34.841657+0000","last_became_active":"2024-09-13T13:29:04.695821+0000","last_became_peered":"2024-09-13T13:29:04.695821+0000","last_unstale":"2024-09-13T13:29:34.841657+0000","last_undegraded":"2024-09-13T13:29:34.841657+0000","last_fullsized":"2024-09-13T13:29:34.841657+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:21:42.012406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.835996+0000","last_change":"2024-09-13T13:29:02.691523+0000","last_active":"2024-09-13T13:29:34.835996+0000","last_peered":"2024-09-13T13:29:34.835996+0000","last_clean":"2024-09-13T13:29:34.835996+0000","last_became_active":"2024-09-13T13:29:02.691189+0000","last_became_peered":"2024-09-13T13:29:02.691189+0000","last_unstale":"2024-09-13T13:29:34.835996+0000","last_undegraded":"2024-09-13T13:29:34.835996+0000","last_fullsized":"2024-09-13T13:29:34.835996+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:28:55.007103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":40,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644495+0000","last_change":"2024-09-13T13:29:04.685741+0000","last_active":"2024-09-13T13:29:34.644495+0000","last_peered":"2024-09-13T13:29:34.644495+0000","last_clean":"2024-09-13T13:29:34.644495+0000","last_became_active":"2024-09-13T13:29:01.674108+0000","last_became_peered":"2024-09-13T13:29:01.674108+0000","last_unstale":"2024-09-13T13:29:34.644495+0000","last_undegraded":"2024-09-13T13:29:34.644495+0000","last_fullsized":"2024-09-13T13:29:34.644495+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_clean_scrub_stamp":"2024-09-13T13:29:00.658664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:21.034317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141281,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837851+0000","last_change":"2024-09-13T13:29:06.700792+0000","last_active":"2024-09-13T13:29:34.837851+0000","last_peered":"2024-09-13T13:29:34.837851+0000","last_clean":"2024-09-13T13:29:34.837851+0000","last_became_active":"2024-09-13T13:29:06.700296+0000","last_became_peered":"2024-09-13T13:29:06.700296+0000","last_unstale":"2024-09-13T13:29:34.837851+0000","last_undegraded":"2024-09-13T13:29:34.837851+0000","last_fullsized":"2024-09-13T13:29:34.837851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:46.034615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155092+0000","last_change":"2024-09-13T13:29:08.716827+0000","last_active":"2024-09-13T13:29:34.155092+0000","last_peered":"2024-09-13T13:29:34.155092+0000","last_clean":"2024-09-13T13:29:34.155092+0000","last_became_active":"2024-09-13T13:29:08.716184+0000","last_became_peered":"2024-09-13T13:29:08.716184+0000","last_unstale":"2024-09-13T13:29:34.155092+0000","last_undegraded":"2024-09-13T13:29:34.155092+0000","last_fullsized":"2024-09-13T13:29:34.155092+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:52.346008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'13","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841965+0000","last_change":"2024-09-13T13:29:04.686667+0000","last_active":"2024-09-13T13:29:34.841965+0000","last_peered":"2024-09-13T13:29:34.841965+0000","last_clean":"2024-09-13T13:29:34.841965+0000","last_became_active":"2024-09-13T13:29:04.686350+0000","last_became_peered":"2024-09-13T13:29:04.686350+0000","last_unstale":"2024-09-13T13:29:34.841965+0000","last_undegraded":"2024-09-13T13:29:34.841965+0000","last_fullsized":"2024-09-13T13:29:34.841965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:43.433752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836237+0000","last_change":"2024-09-13T13:29:02.681520+0000","last_active":"2024-09-13T13:29:34.836237+0000","last_peered":"2024-09-13T13:29:34.836237+0000","last_clean":"2024-09-13T13:29:34.836237+0000","last_became_active":"2024-09-13T13:29:02.679776+0000","last_became_peered":"2024-09-13T13:29:02.679776+0000","last_unstale":"2024-09-13T13:29:34.836237+0000","last_undegraded":"2024-09-13T13:29:34.836237+0000","last_fullsized":"2024-09-13T13:29:34.836237+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:48.793912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157404+0000","last_change":"2024-09-13T13:29:04.686546+0000","last_active":"2024-09-13T13:29:34.157404+0000","last_peered":"2024-09-13T13:29:34.157404+0000","last_clean":"2024-09-13T13:29:34.157404+0000","last_became_active":"2024-09-13T13:29:01.675417+0000","last_became_peered":"2024-09-13T13:29:01.675417+0000","last_unstale":"2024-09-13T13:29:34.157404+0000","last_undegraded":"2024-09-13T13:29:34.157404+0000","last_fullsized":"2024-09-13T13:29:34.157404+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_clean_scrub_stamp":"2024-09-13T13:29:00.658664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:11.816860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088590200000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158162+0000","last_change":"2024-09-13T13:29:06.698984+0000","last_active":"2024-09-13T13:29:34.158162+0000","last_peered":"2024-09-13T13:29:34.158162+0000","last_clean":"2024-09-13T13:29:34.158162+0000","last_became_active":"2024-09-13T13:29:06.698151+0000","last_became_peered":"2024-09-13T13:29:06.698151+0000","last_unstale":"2024-09-13T13:29:34.158162+0000","last_undegraded":"2024-09-13T13:29:34.158162+0000","last_fullsized":"2024-09-13T13:29:34.158162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:31.909616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154477+0000","last_change":"2024-09-13T13:29:08.718127+0000","last_active":"2024-09-13T13:29:34.154477+0000","last_peered":"2024-09-13T13:29:34.154477+0000","last_clean":"2024-09-13T13:29:34.154477+0000","last_became_active":"2024-09-13T13:29:08.717762+0000","last_became_peered":"2024-09-13T13:29:08.717762+0000","last_unstale":"2024-09-13T13:29:34.154477+0000","last_undegraded":"2024-09-13T13:29:34.154477+0000","last_fullsized":"2024-09-13T13:29:34.154477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:55:32.298438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"61'12","reported_seq":44,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839661+0000","last_change":"2024-09-13T13:29:04.691170+0000","last_active":"2024-09-13T13:29:34.839661+0000","last_peered":"2024-09-13T13:29:34.839661+0000","last_clean":"2024-09-13T13:29:34.839661+0000","last_became_active":"2024-09-13T13:29:04.690864+0000","last_became_peered":"2024-09-13T13:29:04.690864+0000","last_unstale":"2024-09-13T13:29:34.839661+0000","last_undegraded":"2024-09-13T13:29:34.839661+0000","last_fullsized":"2024-09-13T13:29:34.839661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:48.140171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838943+0000","last_change":"2024-09-13T13:29:02.693714+0000","last_active":"2024-09-13T13:29:34.838943+0000","last_peered":"2024-09-13T13:29:34.838943+0000","last_clean":"2024-09-13T13:29:34.838943+0000","last_became_active":"2024-09-13T13:29:02.693382+0000","last_became_peered":"2024-09-13T13:29:02.693382+0000","last_unstale":"2024-09-13T13:29:34.838943+0000","last_undegraded":"2024-09-13T13:29:34.838943+0000","last_fullsized":"2024-09-13T13:29:34.838943+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:39:58.373355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"62'5","reported_seq":73,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:38.765759+0000","last_change":"2024-09-13T13:29:04.799476+0000","last_active":"2024-09-13T13:29:38.765759+0000","last_peered":"2024-09-13T13:29:38.765759+0000","last_clean":"2024-09-13T13:29:38.765759+0000","last_became_active":"2024-09-13T13:29:01.674373+0000","last_became_peered":"2024-09-13T13:29:01.674373+0000","last_unstale":"2024-09-13T13:29:38.765759+0000","last_undegraded":"2024-09-13T13:29:38.765759+0000","last_fullsized":"2024-09-13T13:29:38.765759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_clean_scrub_stamp":"2024-09-13T13:29:00.658664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:55.911823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047758099999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644930+0000","last_change":"2024-09-13T13:29:06.703315+0000","last_active":"2024-09-13T13:29:34.644930+0000","last_peered":"2024-09-13T13:29:34.644930+0000","last_clean":"2024-09-13T13:29:34.644930+0000","last_became_active":"2024-09-13T13:29:06.703034+0000","last_became_peered":"2024-09-13T13:29:06.703034+0000","last_unstale":"2024-09-13T13:29:34.644930+0000","last_undegraded":"2024-09-13T13:29:34.644930+0000","last_fullsized":"2024-09-13T13:29:34.644930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:41.495911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841250+0000","last_change":"2024-09-13T13:29:08.715823+0000","last_active":"2024-09-13T13:29:34.841250+0000","last_peered":"2024-09-13T13:29:34.841250+0000","last_clean":"2024-09-13T13:29:34.841250+0000","last_became_active":"2024-09-13T13:29:08.715462+0000","last_became_peered":"2024-09-13T13:29:08.715462+0000","last_unstale":"2024-09-13T13:29:34.841250+0000","last_undegraded":"2024-09-13T13:29:34.841250+0000","last_fullsized":"2024-09-13T13:29:34.841250+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:43:19.382580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'16","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493538+0000","last_change":"2024-09-13T13:29:04.800994+0000","last_active":"2024-09-13T13:29:34.493538+0000","last_peered":"2024-09-13T13:29:34.493538+0000","last_clean":"2024-09-13T13:29:34.493538+0000","last_became_active":"2024-09-13T13:29:04.800600+0000","last_became_peered":"2024-09-13T13:29:04.800600+0000","last_unstale":"2024-09-13T13:29:34.493538+0000","last_undegraded":"2024-09-13T13:29:34.493538+0000","last_fullsized":"2024-09-13T13:29:34.493538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:32.426534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153677+0000","last_change":"2024-09-13T13:29:02.676473+0000","last_active":"2024-09-13T13:29:34.153677+0000","last_peered":"2024-09-13T13:29:34.153677+0000","last_clean":"2024-09-13T13:29:34.153677+0000","last_became_active":"2024-09-13T13:29:02.676183+0000","last_became_peered":"2024-09-13T13:29:02.676183+0000","last_unstale":"2024-09-13T13:29:34.153677+0000","last_undegraded":"2024-09-13T13:29:34.153677+0000","last_fullsized":"2024-09-13T13:29:34.153677+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:54.804101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"63'79","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:36.623541+0000","last_change":"2024-09-13T13:28:55.532878+0000","last_active":"2024-09-13T13:29:36.623541+0000","last_peered":"2024-09-13T13:29:36.623541+0000","last_clean":"2024-09-13T13:29:36.623541+0000","last_became_active":"2024-09-13T13:28:55.527403+0000","last_became_peered":"2024-09-13T13:28:55.527403+0000","last_unstale":"2024-09-13T13:29:36.623541+0000","last_undegraded":"2024-09-13T13:29:36.623541+0000","last_fullsized":"2024-09-13T13:29:36.623541+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:26:28.508497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:26:28.508497+0000","last_clean_scrub_stamp":"2024-09-13T13:26:28.508497+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:40.153961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154320+0000","last_change":"2024-09-13T13:29:06.698751+0000","last_active":"2024-09-13T13:29:34.154320+0000","last_peered":"2024-09-13T13:29:34.154320+0000","last_clean":"2024-09-13T13:29:34.154320+0000","last_became_active":"2024-09-13T13:29:06.698371+0000","last_became_peered":"2024-09-13T13:29:06.698371+0000","last_unstale":"2024-09-13T13:29:34.154320+0000","last_undegraded":"2024-09-13T13:29:34.154320+0000","last_fullsized":"2024-09-13T13:29:34.154320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:01.079661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644801+0000","last_change":"2024-09-13T13:29:08.715234+0000","last_active":"2024-09-13T13:29:34.644801+0000","last_peered":"2024-09-13T13:29:34.644801+0000","last_clean":"2024-09-13T13:29:34.644801+0000","last_became_active":"2024-09-13T13:29:08.714949+0000","last_became_peered":"2024-09-13T13:29:08.714949+0000","last_unstale":"2024-09-13T13:29:34.644801+0000","last_undegraded":"2024-09-13T13:29:34.644801+0000","last_fullsized":"2024-09-13T13:29:34.644801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:11.765018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837083+0000","last_change":"2024-09-13T13:29:04.696280+0000","last_active":"2024-09-13T13:29:34.837083+0000","last_peered":"2024-09-13T13:29:34.837083+0000","last_clean":"2024-09-13T13:29:34.837083+0000","last_became_active":"2024-09-13T13:29:04.695969+0000","last_became_peered":"2024-09-13T13:29:04.695969+0000","last_unstale":"2024-09-13T13:29:34.837083+0000","last_undegraded":"2024-09-13T13:29:34.837083+0000","last_fullsized":"2024-09-13T13:29:34.837083+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:28:38.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.835958+0000","last_change":"2024-09-13T13:29:02.680615+0000","last_active":"2024-09-13T13:29:34.835958+0000","last_peered":"2024-09-13T13:29:34.835958+0000","last_clean":"2024-09-13T13:29:34.835958+0000","last_became_active":"2024-09-13T13:29:02.680308+0000","last_became_peered":"2024-09-13T13:29:02.680308+0000","last_unstale":"2024-09-13T13:29:34.835958+0000","last_undegraded":"2024-09-13T13:29:34.835958+0000","last_fullsized":"2024-09-13T13:29:34.835958+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:33.074924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644757+0000","last_change":"2024-09-13T13:29:06.697801+0000","last_active":"2024-09-13T13:29:34.644757+0000","last_peered":"2024-09-13T13:29:34.644757+0000","last_clean":"2024-09-13T13:29:34.644757+0000","last_became_active":"2024-09-13T13:29:06.697499+0000","last_became_peered":"2024-09-13T13:29:06.697499+0000","last_unstale":"2024-09-13T13:29:34.644757+0000","last_undegraded":"2024-09-13T13:29:34.644757+0000","last_fullsized":"2024-09-13T13:29:34.644757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:54.456923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153210+0000","last_change":"2024-09-13T13:29:08.707814+0000","last_active":"2024-09-13T13:29:34.153210+0000","last_peered":"2024-09-13T13:29:34.153210+0000","last_clean":"2024-09-13T13:29:34.153210+0000","last_became_active":"2024-09-13T13:29:08.707486+0000","last_became_peered":"2024-09-13T13:29:08.707486+0000","last_unstale":"2024-09-13T13:29:34.153210+0000","last_undegraded":"2024-09-13T13:29:34.153210+0000","last_fullsized":"2024-09-13T13:29:34.153210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:00.243537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"61'17","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837116+0000","last_change":"2024-09-13T13:29:04.698821+0000","last_active":"2024-09-13T13:29:34.837116+0000","last_peered":"2024-09-13T13:29:34.837116+0000","last_clean":"2024-09-13T13:29:34.837116+0000","last_became_active":"2024-09-13T13:29:04.698188+0000","last_became_peered":"2024-09-13T13:29:04.698188+0000","last_unstale":"2024-09-13T13:29:34.837116+0000","last_undegraded":"2024-09-13T13:29:34.837116+0000","last_fullsized":"2024-09-13T13:29:34.837116+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:00.483577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493609+0000","last_change":"2024-09-13T13:29:02.678254+0000","last_active":"2024-09-13T13:29:34.493609+0000","last_peered":"2024-09-13T13:29:34.493609+0000","last_clean":"2024-09-13T13:29:34.493609+0000","last_became_active":"2024-09-13T13:29:02.677913+0000","last_became_peered":"2024-09-13T13:29:02.677913+0000","last_unstale":"2024-09-13T13:29:34.493609+0000","last_undegraded":"2024-09-13T13:29:34.493609+0000","last_fullsized":"2024-09-13T13:29:34.493609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:45.979879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841366+0000","last_change":"2024-09-13T13:29:06.701990+0000","last_active":"2024-09-13T13:29:34.841366+0000","last_peered":"2024-09-13T13:29:34.841366+0000","last_clean":"2024-09-13T13:29:34.841366+0000","last_became_active":"2024-09-13T13:29:06.701684+0000","last_became_peered":"2024-09-13T13:29:06.701684+0000","last_unstale":"2024-09-13T13:29:34.841366+0000","last_undegraded":"2024-09-13T13:29:34.841366+0000","last_fullsized":"2024-09-13T13:29:34.841366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:12.312236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158388+0000","last_change":"2024-09-13T13:29:08.704382+0000","last_active":"2024-09-13T13:29:34.158388+0000","last_peered":"2024-09-13T13:29:34.158388+0000","last_clean":"2024-09-13T13:29:34.158388+0000","last_became_active":"2024-09-13T13:29:08.704028+0000","last_became_peered":"2024-09-13T13:29:08.704028+0000","last_unstale":"2024-09-13T13:29:34.158388+0000","last_undegraded":"2024-09-13T13:29:34.158388+0000","last_fullsized":"2024-09-13T13:29:34.158388+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:02:03.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836913+0000","last_change":"2024-09-13T13:29:04.698667+0000","last_active":"2024-09-13T13:29:34.836913+0000","last_peered":"2024-09-13T13:29:34.836913+0000","last_clean":"2024-09-13T13:29:34.836913+0000","last_became_active":"2024-09-13T13:29:04.695505+0000","last_became_peered":"2024-09-13T13:29:04.695505+0000","last_unstale":"2024-09-13T13:29:34.836913+0000","last_undegraded":"2024-09-13T13:29:34.836913+0000","last_fullsized":"2024-09-13T13:29:34.836913+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:13.192158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153532+0000","last_change":"2024-09-13T13:29:02.689313+0000","last_active":"2024-09-13T13:29:34.153532+0000","last_peered":"2024-09-13T13:29:34.153532+0000","last_clean":"2024-09-13T13:29:34.153532+0000","last_became_active":"2024-09-13T13:29:02.688724+0000","last_became_peered":"2024-09-13T13:29:02.688724+0000","last_unstale":"2024-09-13T13:29:34.153532+0000","last_undegraded":"2024-09-13T13:29:34.153532+0000","last_fullsized":"2024-09-13T13:29:34.153532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:31.644591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"61'8","reported_seq":38,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157784+0000","last_change":"2024-09-13T13:29:06.696961+0000","last_active":"2024-09-13T13:29:34.157784+0000","last_peered":"2024-09-13T13:29:34.157784+0000","last_clean":"2024-09-13T13:29:34.157784+0000","last_became_active":"2024-09-13T13:29:06.696555+0000","last_became_peered":"2024-09-13T13:29:06.696555+0000","last_unstale":"2024-09-13T13:29:34.157784+0000","last_undegraded":"2024-09-13T13:29:34.157784+0000","last_fullsized":"2024-09-13T13:29:34.157784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:53:44.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836846+0000","last_change":"2024-09-13T13:29:08.710508+0000","last_active":"2024-09-13T13:29:34.836846+0000","last_peered":"2024-09-13T13:29:34.836846+0000","last_clean":"2024-09-13T13:29:34.836846+0000","last_became_active":"2024-09-13T13:29:08.710216+0000","last_became_peered":"2024-09-13T13:29:08.710216+0000","last_unstale":"2024-09-13T13:29:34.836846+0000","last_undegraded":"2024-09-13T13:29:34.836846+0000","last_fullsized":"2024-09-13T13:29:34.836846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:56.777900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.840431+0000","last_change":"2024-09-13T13:29:04.692866+0000","last_active":"2024-09-13T13:29:34.840431+0000","last_peered":"2024-09-13T13:29:34.840431+0000","last_clean":"2024-09-13T13:29:34.840431+0000","last_became_active":"2024-09-13T13:29:04.692526+0000","last_became_peered":"2024-09-13T13:29:04.692526+0000","last_unstale":"2024-09-13T13:29:34.840431+0000","last_undegraded":"2024-09-13T13:29:34.840431+0000","last_fullsized":"2024-09-13T13:29:34.840431+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:46.971234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644650+0000","last_change":"2024-09-13T13:29:02.681203+0000","last_active":"2024-09-13T13:29:34.644650+0000","last_peered":"2024-09-13T13:29:34.644650+0000","last_clean":"2024-09-13T13:29:34.644650+0000","last_became_active":"2024-09-13T13:29:02.680424+0000","last_became_peered":"2024-09-13T13:29:02.680424+0000","last_unstale":"2024-09-13T13:29:34.644650+0000","last_undegraded":"2024-09-13T13:29:34.644650+0000","last_fullsized":"2024-09-13T13:29:34.644650+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.803414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157811+0000","last_change":"2024-09-13T13:29:06.700622+0000","last_active":"2024-09-13T13:29:34.157811+0000","last_peered":"2024-09-13T13:29:34.157811+0000","last_clean":"2024-09-13T13:29:34.157811+0000","last_became_active":"2024-09-13T13:29:06.700210+0000","last_became_peered":"2024-09-13T13:29:06.700210+0000","last_unstale":"2024-09-13T13:29:34.157811+0000","last_undegraded":"2024-09-13T13:29:34.157811+0000","last_fullsized":"2024-09-13T13:29:34.157811+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:19.443785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839101+0000","last_change":"2024-09-13T13:29:08.712559+0000","last_active":"2024-09-13T13:29:34.839101+0000","last_peered":"2024-09-13T13:29:34.839101+0000","last_clean":"2024-09-13T13:29:34.839101+0000","last_became_active":"2024-09-13T13:29:08.712260+0000","last_became_peered":"2024-09-13T13:29:08.712260+0000","last_unstale":"2024-09-13T13:29:34.839101+0000","last_undegraded":"2024-09-13T13:29:34.839101+0000","last_fullsized":"2024-09-13T13:29:34.839101+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:57.465810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'19","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493264+0000","last_change":"2024-09-13T13:29:04.799952+0000","last_active":"2024-09-13T13:29:34.493264+0000","last_peered":"2024-09-13T13:29:34.493264+0000","last_clean":"2024-09-13T13:29:34.493264+0000","last_became_active":"2024-09-13T13:29:04.799571+0000","last_became_peered":"2024-09-13T13:29:04.799571+0000","last_unstale":"2024-09-13T13:29:34.493264+0000","last_undegraded":"2024-09-13T13:29:34.493264+0000","last_fullsized":"2024-09-13T13:29:34.493264+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:01.616633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154725+0000","last_change":"2024-09-13T13:29:02.680276+0000","last_active":"2024-09-13T13:29:34.154725+0000","last_peered":"2024-09-13T13:29:34.154725+0000","last_clean":"2024-09-13T13:29:34.154725+0000","last_became_active":"2024-09-13T13:29:02.679923+0000","last_became_peered":"2024-09-13T13:29:02.679923+0000","last_unstale":"2024-09-13T13:29:34.154725+0000","last_undegraded":"2024-09-13T13:29:34.154725+0000","last_fullsized":"2024-09-13T13:29:34.154725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:55:02.901598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157564+0000","last_change":"2024-09-13T13:29:06.703939+0000","last_active":"2024-09-13T13:29:34.157564+0000","last_peered":"2024-09-13T13:29:34.157564+0000","last_clean":"2024-09-13T13:29:34.157564+0000","last_became_active":"2024-09-13T13:29:06.702963+0000","last_became_peered":"2024-09-13T13:29:06.702963+0000","last_unstale":"2024-09-13T13:29:34.157564+0000","last_undegraded":"2024-09-13T13:29:34.157564+0000","last_fullsized":"2024-09-13T13:29:34.157564+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:41:40.625430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153914+0000","last_change":"2024-09-13T13:29:08.716797+0000","last_active":"2024-09-13T13:29:34.153914+0000","last_peered":"2024-09-13T13:29:34.153914+0000","last_clean":"2024-09-13T13:29:34.153914+0000","last_became_active":"2024-09-13T13:29:08.716518+0000","last_became_peered":"2024-09-13T13:29:08.716518+0000","last_unstale":"2024-09-13T13:29:34.153914+0000","last_undegraded":"2024-09-13T13:29:34.153914+0000","last_fullsized":"2024-09-13T13:29:34.153914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:12.815349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836648+0000","last_change":"2024-09-13T13:29:04.697964+0000","last_active":"2024-09-13T13:29:34.836648+0000","last_peered":"2024-09-13T13:29:34.836648+0000","last_clean":"2024-09-13T13:29:34.836648+0000","last_became_active":"2024-09-13T13:29:04.697071+0000","last_became_peered":"2024-09-13T13:29:04.697071+0000","last_unstale":"2024-09-13T13:29:34.836648+0000","last_undegraded":"2024-09-13T13:29:34.836648+0000","last_fullsized":"2024-09-13T13:29:34.836648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:03.248740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154769+0000","last_change":"2024-09-13T13:29:02.685296+0000","last_active":"2024-09-13T13:29:34.154769+0000","last_peered":"2024-09-13T13:29:34.154769+0000","last_clean":"2024-09-13T13:29:34.154769+0000","last_became_active":"2024-09-13T13:29:02.684915+0000","last_became_peered":"2024-09-13T13:29:02.684915+0000","last_unstale":"2024-09-13T13:29:34.154769+0000","last_undegraded":"2024-09-13T13:29:34.154769+0000","last_fullsized":"2024-09-13T13:29:34.154769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:35.594765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157718+0000","last_change":"2024-09-13T13:29:06.699902+0000","last_active":"2024-09-13T13:29:34.157718+0000","last_peered":"2024-09-13T13:29:34.157718+0000","last_clean":"2024-09-13T13:29:34.157718+0000","last_became_active":"2024-09-13T13:29:06.699455+0000","last_became_peered":"2024-09-13T13:29:06.699455+0000","last_unstale":"2024-09-13T13:29:34.157718+0000","last_undegraded":"2024-09-13T13:29:34.157718+0000","last_fullsized":"2024-09-13T13:29:34.157718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:23.533761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153305+0000","last_change":"2024-09-13T13:29:08.714941+0000","last_active":"2024-09-13T13:29:34.153305+0000","last_peered":"2024-09-13T13:29:34.153305+0000","last_clean":"2024-09-13T13:29:34.153305+0000","last_became_active":"2024-09-13T13:29:08.714643+0000","last_became_peered":"2024-09-13T13:29:08.714643+0000","last_unstale":"2024-09-13T13:29:34.153305+0000","last_undegraded":"2024-09-13T13:29:34.153305+0000","last_fullsized":"2024-09-13T13:29:34.153305+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:47.476192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644228+0000","last_change":"2024-09-13T13:29:04.799373+0000","last_active":"2024-09-13T13:29:34.644228+0000","last_peered":"2024-09-13T13:29:34.644228+0000","last_clean":"2024-09-13T13:29:34.644228+0000","last_became_active":"2024-09-13T13:29:04.799070+0000","last_became_peered":"2024-09-13T13:29:04.799070+0000","last_unstale":"2024-09-13T13:29:34.644228+0000","last_undegraded":"2024-09-13T13:29:34.644228+0000","last_fullsized":"2024-09-13T13:29:34.644228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:46.513542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"54'2","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154949+0000","last_change":"2024-09-13T13:29:02.689075+0000","last_active":"2024-09-13T13:29:34.154949+0000","last_peered":"2024-09-13T13:29:34.154949+0000","last_clean":"2024-09-13T13:29:34.154949+0000","last_became_active":"2024-09-13T13:29:02.688653+0000","last_became_peered":"2024-09-13T13:29:02.688653+0000","last_unstale":"2024-09-13T13:29:34.154949+0000","last_undegraded":"2024-09-13T13:29:34.154949+0000","last_fullsized":"2024-09-13T13:29:34.154949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:12:07.177401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"61'8","reported_seq":38,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153985+0000","last_change":"2024-09-13T13:29:06.697615+0000","last_active":"2024-09-13T13:29:34.153985+0000","last_peered":"2024-09-13T13:29:34.153985+0000","last_clean":"2024-09-13T13:29:34.153985+0000","last_became_active":"2024-09-13T13:29:06.697354+0000","last_became_peered":"2024-09-13T13:29:06.697354+0000","last_unstale":"2024-09-13T13:29:34.153985+0000","last_undegraded":"2024-09-13T13:29:34.153985+0000","last_fullsized":"2024-09-13T13:29:34.153985+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:46.455383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644284+0000","last_change":"2024-09-13T13:29:08.712561+0000","last_active":"2024-09-13T13:29:34.644284+0000","last_peered":"2024-09-13T13:29:34.644284+0000","last_clean":"2024-09-13T13:29:34.644284+0000","last_became_active":"2024-09-13T13:29:08.712287+0000","last_became_peered":"2024-09-13T13:29:08.712287+0000","last_unstale":"2024-09-13T13:29:34.644284+0000","last_undegraded":"2024-09-13T13:29:34.644284+0000","last_fullsized":"2024-09-13T13:29:34.644284+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:44.552490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493238+0000","last_change":"2024-09-13T13:29:02.688481+0000","last_active":"2024-09-13T13:29:34.493238+0000","last_peered":"2024-09-13T13:29:34.493238+0000","last_clean":"2024-09-13T13:29:34.493238+0000","last_became_active":"2024-09-13T13:29:02.688193+0000","last_became_peered":"2024-09-13T13:29:02.688193+0000","last_unstale":"2024-09-13T13:29:34.493238+0000","last_undegraded":"2024-09-13T13:29:34.493238+0000","last_fullsized":"2024-09-13T13:29:34.493238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:35.246999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"61'6","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155571+0000","last_change":"2024-09-13T13:29:04.800182+0000","last_active":"2024-09-13T13:29:34.155571+0000","last_peered":"2024-09-13T13:29:34.155571+0000","last_clean":"2024-09-13T13:29:34.155571+0000","last_became_active":"2024-09-13T13:29:04.799735+0000","last_became_peered":"2024-09-13T13:29:04.799735+0000","last_unstale":"2024-09-13T13:29:34.155571+0000","last_undegraded":"2024-09-13T13:29:34.155571+0000","last_fullsized":"2024-09-13T13:29:34.155571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:33.977782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644722+0000","last_change":"2024-09-13T13:29:06.699967+0000","last_active":"2024-09-13T13:29:34.644722+0000","last_peered":"2024-09-13T13:29:34.644722+0000","last_clean":"2024-09-13T13:29:34.644722+0000","last_became_active":"2024-09-13T13:29:06.699655+0000","last_became_peered":"2024-09-13T13:29:06.699655+0000","last_unstale":"2024-09-13T13:29:34.644722+0000","last_undegraded":"2024-09-13T13:29:34.644722+0000","last_fullsized":"2024-09-13T13:29:34.644722+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:41.204008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154410+0000","last_change":"2024-09-13T13:29:08.707626+0000","last_active":"2024-09-13T13:29:34.154410+0000","last_peered":"2024-09-13T13:29:34.154410+0000","last_clean":"2024-09-13T13:29:34.154410+0000","last_became_active":"2024-09-13T13:29:08.707305+0000","last_became_peered":"2024-09-13T13:29:08.707305+0000","last_unstale":"2024-09-13T13:29:34.154410+0000","last_undegraded":"2024-09-13T13:29:34.154410+0000","last_fullsized":"2024-09-13T13:29:34.154410+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:25.910973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839793+0000","last_change":"2024-09-13T13:29:04.801476+0000","last_active":"2024-09-13T13:29:34.839793+0000","last_peered":"2024-09-13T13:29:34.839793+0000","last_clean":"2024-09-13T13:29:34.839793+0000","last_became_active":"2024-09-13T13:29:04.801176+0000","last_became_peered":"2024-09-13T13:29:04.801176+0000","last_unstale":"2024-09-13T13:29:34.839793+0000","last_undegraded":"2024-09-13T13:29:34.839793+0000","last_fullsized":"2024-09-13T13:29:34.839793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:12.467846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154049+0000","last_change":"2024-09-13T13:29:02.681133+0000","last_active":"2024-09-13T13:29:34.154049+0000","last_peered":"2024-09-13T13:29:34.154049+0000","last_clean":"2024-09-13T13:29:34.154049+0000","last_became_active":"2024-09-13T13:29:02.680817+0000","last_became_peered":"2024-09-13T13:29:02.680817+0000","last_unstale":"2024-09-13T13:29:34.154049+0000","last_undegraded":"2024-09-13T13:29:34.154049+0000","last_fullsized":"2024-09-13T13:29:34.154049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.350353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155469+0000","last_change":"2024-09-13T13:29:06.701850+0000","last_active":"2024-09-13T13:29:34.155469+0000","last_peered":"2024-09-13T13:29:34.155469+0000","last_clean":"2024-09-13T13:29:34.155469+0000","last_became_active":"2024-09-13T13:29:06.701530+0000","last_became_peered":"2024-09-13T13:29:06.701530+0000","last_unstale":"2024-09-13T13:29:34.155469+0000","last_undegraded":"2024-09-13T13:29:34.155469+0000","last_fullsized":"2024-09-13T13:29:34.155469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:34:39.699409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158045+0000","last_change":"2024-09-13T13:29:08.709476+0000","last_active":"2024-09-13T13:29:34.158045+0000","last_peered":"2024-09-13T13:29:34.158045+0000","last_clean":"2024-09-13T13:29:34.158045+0000","last_became_active":"2024-09-13T13:29:08.709144+0000","last_became_peered":"2024-09-13T13:29:08.709144+0000","last_unstale":"2024-09-13T13:29:34.158045+0000","last_undegraded":"2024-09-13T13:29:34.158045+0000","last_fullsized":"2024-09-13T13:29:34.158045+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:55.180192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644893+0000","last_change":"2024-09-13T13:29:04.801162+0000","last_active":"2024-09-13T13:29:34.644893+0000","last_peered":"2024-09-13T13:29:34.644893+0000","last_clean":"2024-09-13T13:29:34.644893+0000","last_became_active":"2024-09-13T13:29:04.800813+0000","last_became_peered":"2024-09-13T13:29:04.800813+0000","last_unstale":"2024-09-13T13:29:34.644893+0000","last_undegraded":"2024-09-13T13:29:34.644893+0000","last_fullsized":"2024-09-13T13:29:34.644893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:00.728217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837956+0000","last_change":"2024-09-13T13:29:02.689002+0000","last_active":"2024-09-13T13:29:34.837956+0000","last_peered":"2024-09-13T13:29:34.837956+0000","last_clean":"2024-09-13T13:29:34.837956+0000","last_became_active":"2024-09-13T13:29:02.688591+0000","last_became_peered":"2024-09-13T13:29:02.688591+0000","last_unstale":"2024-09-13T13:29:34.837956+0000","last_undegraded":"2024-09-13T13:29:34.837956+0000","last_fullsized":"2024-09-13T13:29:34.837956+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:20.535245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"61'8","reported_seq":38,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153644+0000","last_change":"2024-09-13T13:29:06.699492+0000","last_active":"2024-09-13T13:29:34.153644+0000","last_peered":"2024-09-13T13:29:34.153644+0000","last_clean":"2024-09-13T13:29:34.153644+0000","last_became_active":"2024-09-13T13:29:06.699160+0000","last_became_peered":"2024-09-13T13:29:06.699160+0000","last_unstale":"2024-09-13T13:29:34.153644+0000","last_undegraded":"2024-09-13T13:29:34.153644+0000","last_fullsized":"2024-09-13T13:29:34.153644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:37.261283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837130+0000","last_change":"2024-09-13T13:29:08.705412+0000","last_active":"2024-09-13T13:29:34.837130+0000","last_peered":"2024-09-13T13:29:34.837130+0000","last_clean":"2024-09-13T13:29:34.837130+0000","last_became_active":"2024-09-13T13:29:08.705139+0000","last_became_peered":"2024-09-13T13:29:08.705139+0000","last_unstale":"2024-09-13T13:29:34.837130+0000","last_undegraded":"2024-09-13T13:29:34.837130+0000","last_fullsized":"2024-09-13T13:29:34.837130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:01:01.510825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154573+0000","last_change":"2024-09-13T13:29:04.799995+0000","last_active":"2024-09-13T13:29:34.154573+0000","last_peered":"2024-09-13T13:29:34.154573+0000","last_clean":"2024-09-13T13:29:34.154573+0000","last_became_active":"2024-09-13T13:29:04.799606+0000","last_became_peered":"2024-09-13T13:29:04.799606+0000","last_unstale":"2024-09-13T13:29:34.154573+0000","last_undegraded":"2024-09-13T13:29:34.154573+0000","last_fullsized":"2024-09-13T13:29:34.154573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:58.222622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154218+0000","last_change":"2024-09-13T13:29:02.683309+0000","last_active":"2024-09-13T13:29:34.154218+0000","last_peered":"2024-09-13T13:29:34.154218+0000","last_clean":"2024-09-13T13:29:34.154218+0000","last_became_active":"2024-09-13T13:29:02.683000+0000","last_became_peered":"2024-09-13T13:29:02.683000+0000","last_unstale":"2024-09-13T13:29:34.154218+0000","last_undegraded":"2024-09-13T13:29:34.154218+0000","last_fullsized":"2024-09-13T13:29:34.154218+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:42.278169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"61'8","reported_seq":38,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644822+0000","last_change":"2024-09-13T13:29:06.703646+0000","last_active":"2024-09-13T13:29:34.644822+0000","last_peered":"2024-09-13T13:29:34.644822+0000","last_clean":"2024-09-13T13:29:34.644822+0000","last_became_active":"2024-09-13T13:29:06.703054+0000","last_became_peered":"2024-09-13T13:29:06.703054+0000","last_unstale":"2024-09-13T13:29:34.644822+0000","last_undegraded":"2024-09-13T13:29:34.644822+0000","last_fullsized":"2024-09-13T13:29:34.644822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:26.234583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839327+0000","last_change":"2024-09-13T13:29:08.705234+0000","last_active":"2024-09-13T13:29:34.839327+0000","last_peered":"2024-09-13T13:29:34.839327+0000","last_clean":"2024-09-13T13:29:34.839327+0000","last_became_active":"2024-09-13T13:29:08.704922+0000","last_became_peered":"2024-09-13T13:29:08.704922+0000","last_unstale":"2024-09-13T13:29:34.839327+0000","last_undegraded":"2024-09-13T13:29:34.839327+0000","last_fullsized":"2024-09-13T13:29:34.839327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:53.407453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836675+0000","last_change":"2024-09-13T13:29:04.696565+0000","last_active":"2024-09-13T13:29:34.836675+0000","last_peered":"2024-09-13T13:29:34.836675+0000","last_clean":"2024-09-13T13:29:34.836675+0000","last_became_active":"2024-09-13T13:29:04.696241+0000","last_became_peered":"2024-09-13T13:29:04.696241+0000","last_unstale":"2024-09-13T13:29:34.836675+0000","last_undegraded":"2024-09-13T13:29:34.836675+0000","last_fullsized":"2024-09-13T13:29:34.836675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:50.681411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836007+0000","last_change":"2024-09-13T13:29:02.680918+0000","last_active":"2024-09-13T13:29:34.836007+0000","last_peered":"2024-09-13T13:29:34.836007+0000","last_clean":"2024-09-13T13:29:34.836007+0000","last_became_active":"2024-09-13T13:29:02.680654+0000","last_became_peered":"2024-09-13T13:29:02.680654+0000","last_unstale":"2024-09-13T13:29:34.836007+0000","last_undegraded":"2024-09-13T13:29:34.836007+0000","last_fullsized":"2024-09-13T13:29:34.836007+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:38:55.163478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837677+0000","last_change":"2024-09-13T13:29:06.699233+0000","last_active":"2024-09-13T13:29:34.837677+0000","last_peered":"2024-09-13T13:29:34.837677+0000","last_clean":"2024-09-13T13:29:34.837677+0000","last_became_active":"2024-09-13T13:29:06.698750+0000","last_became_peered":"2024-09-13T13:29:06.698750+0000","last_unstale":"2024-09-13T13:29:34.837677+0000","last_undegraded":"2024-09-13T13:29:34.837677+0000","last_fullsized":"2024-09-13T13:29:34.837677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:18:52.021184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152778+0000","last_change":"2024-09-13T13:29:08.713644+0000","last_active":"2024-09-13T13:29:34.152778+0000","last_peered":"2024-09-13T13:29:34.152778+0000","last_clean":"2024-09-13T13:29:34.152778+0000","last_became_active":"2024-09-13T13:29:08.709970+0000","last_became_peered":"2024-09-13T13:29:08.709970+0000","last_unstale":"2024-09-13T13:29:34.152778+0000","last_undegraded":"2024-09-13T13:29:34.152778+0000","last_fullsized":"2024-09-13T13:29:34.152778+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:13.722611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841793+0000","last_change":"2024-09-13T13:29:04.694199+0000","last_active":"2024-09-13T13:29:34.841793+0000","last_peered":"2024-09-13T13:29:34.841793+0000","last_clean":"2024-09-13T13:29:34.841793+0000","last_became_active":"2024-09-13T13:29:04.693855+0000","last_became_peered":"2024-09-13T13:29:04.693855+0000","last_unstale":"2024-09-13T13:29:34.841793+0000","last_undegraded":"2024-09-13T13:29:34.841793+0000","last_fullsized":"2024-09-13T13:29:34.841793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:32.409401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154630+0000","last_change":"2024-09-13T13:29:02.686454+0000","last_active":"2024-09-13T13:29:34.154630+0000","last_peered":"2024-09-13T13:29:34.154630+0000","last_clean":"2024-09-13T13:29:34.154630+0000","last_became_active":"2024-09-13T13:29:02.686126+0000","last_became_peered":"2024-09-13T13:29:02.686126+0000","last_unstale":"2024-09-13T13:29:34.154630+0000","last_undegraded":"2024-09-13T13:29:34.154630+0000","last_fullsized":"2024-09-13T13:29:34.154630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:08.201784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153372+0000","last_change":"2024-09-13T13:29:06.702382+0000","last_active":"2024-09-13T13:29:34.153372+0000","last_peered":"2024-09-13T13:29:34.153372+0000","last_clean":"2024-09-13T13:29:34.153372+0000","last_became_active":"2024-09-13T13:29:06.702042+0000","last_became_peered":"2024-09-13T13:29:06.702042+0000","last_unstale":"2024-09-13T13:29:34.153372+0000","last_undegraded":"2024-09-13T13:29:34.153372+0000","last_fullsized":"2024-09-13T13:29:34.153372+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:23:37.373822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838435+0000","last_change":"2024-09-13T13:29:08.711167+0000","last_active":"2024-09-13T13:29:34.838435+0000","last_peered":"2024-09-13T13:29:34.838435+0000","last_clean":"2024-09-13T13:29:34.838435+0000","last_became_active":"2024-09-13T13:29:08.710865+0000","last_became_peered":"2024-09-13T13:29:08.710865+0000","last_unstale":"2024-09-13T13:29:34.838435+0000","last_undegraded":"2024-09-13T13:29:34.838435+0000","last_fullsized":"2024-09-13T13:29:34.838435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:32.530162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152879+0000","last_change":"2024-09-13T13:29:04.801077+0000","last_active":"2024-09-13T13:29:34.152879+0000","last_peered":"2024-09-13T13:29:34.152879+0000","last_clean":"2024-09-13T13:29:34.152879+0000","last_became_active":"2024-09-13T13:29:04.800438+0000","last_became_peered":"2024-09-13T13:29:04.800438+0000","last_unstale":"2024-09-13T13:29:34.152879+0000","last_undegraded":"2024-09-13T13:29:34.152879+0000","last_fullsized":"2024-09-13T13:29:34.152879+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:22.074292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644834+0000","last_change":"2024-09-13T13:29:02.684061+0000","last_active":"2024-09-13T13:29:34.644834+0000","last_peered":"2024-09-13T13:29:34.644834+0000","last_clean":"2024-09-13T13:29:34.644834+0000","last_became_active":"2024-09-13T13:29:02.683750+0000","last_became_peered":"2024-09-13T13:29:02.683750+0000","last_unstale":"2024-09-13T13:29:34.644834+0000","last_undegraded":"2024-09-13T13:29:34.644834+0000","last_fullsized":"2024-09-13T13:29:34.644834+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:19.345241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153649+0000","last_change":"2024-09-13T13:29:06.700724+0000","last_active":"2024-09-13T13:29:34.153649+0000","last_peered":"2024-09-13T13:29:34.153649+0000","last_clean":"2024-09-13T13:29:34.153649+0000","last_became_active":"2024-09-13T13:29:06.700441+0000","last_became_peered":"2024-09-13T13:29:06.700441+0000","last_unstale":"2024-09-13T13:29:34.153649+0000","last_undegraded":"2024-09-13T13:29:34.153649+0000","last_fullsized":"2024-09-13T13:29:34.153649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:43.104223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153501+0000","last_change":"2024-09-13T13:29:08.705182+0000","last_active":"2024-09-13T13:29:34.153501+0000","last_peered":"2024-09-13T13:29:34.153501+0000","last_clean":"2024-09-13T13:29:34.153501+0000","last_became_active":"2024-09-13T13:29:08.704848+0000","last_became_peered":"2024-09-13T13:29:08.704848+0000","last_unstale":"2024-09-13T13:29:34.153501+0000","last_undegraded":"2024-09-13T13:29:34.153501+0000","last_fullsized":"2024-09-13T13:29:34.153501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:39.104724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"61'4","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152753+0000","last_change":"2024-09-13T13:29:04.697197+0000","last_active":"2024-09-13T13:29:34.152753+0000","last_peered":"2024-09-13T13:29:34.152753+0000","last_clean":"2024-09-13T13:29:34.152753+0000","last_became_active":"2024-09-13T13:29:04.696858+0000","last_became_peered":"2024-09-13T13:29:04.696858+0000","last_unstale":"2024-09-13T13:29:34.152753+0000","last_undegraded":"2024-09-13T13:29:34.152753+0000","last_fullsized":"2024-09-13T13:29:34.152753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:49.140079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836989+0000","last_change":"2024-09-13T13:29:02.691306+0000","last_active":"2024-09-13T13:29:34.836989+0000","last_peered":"2024-09-13T13:29:34.836989+0000","last_clean":"2024-09-13T13:29:34.836989+0000","last_became_active":"2024-09-13T13:29:02.690950+0000","last_became_peered":"2024-09-13T13:29:02.690950+0000","last_unstale":"2024-09-13T13:29:34.836989+0000","last_undegraded":"2024-09-13T13:29:34.836989+0000","last_fullsized":"2024-09-13T13:29:34.836989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493157+0000","last_change":"2024-09-13T13:29:06.701062+0000","last_active":"2024-09-13T13:29:34.493157+0000","last_peered":"2024-09-13T13:29:34.493157+0000","last_clean":"2024-09-13T13:29:34.493157+0000","last_became_active":"2024-09-13T13:29:06.700697+0000","last_became_peered":"2024-09-13T13:29:06.700697+0000","last_unstale":"2024-09-13T13:29:34.493157+0000","last_undegraded":"2024-09-13T13:29:34.493157+0000","last_fullsized":"2024-09-13T13:29:34.493157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:55.044019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153821+0000","last_change":"2024-09-13T13:29:08.708426+0000","last_active":"2024-09-13T13:29:34.153821+0000","last_peered":"2024-09-13T13:29:34.153821+0000","last_clean":"2024-09-13T13:29:34.153821+0000","last_became_active":"2024-09-13T13:29:08.708161+0000","last_became_peered":"2024-09-13T13:29:08.708161+0000","last_unstale":"2024-09-13T13:29:34.153821+0000","last_undegraded":"2024-09-13T13:29:34.153821+0000","last_fullsized":"2024-09-13T13:29:34.153821+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:16.692455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837712+0000","last_change":"2024-09-13T13:29:08.714777+0000","last_active":"2024-09-13T13:29:34.837712+0000","last_peered":"2024-09-13T13:29:34.837712+0000","last_clean":"2024-09-13T13:29:34.837712+0000","last_became_active":"2024-09-13T13:29:08.714406+0000","last_became_peered":"2024-09-13T13:29:08.714406+0000","last_unstale":"2024-09-13T13:29:34.837712+0000","last_undegraded":"2024-09-13T13:29:34.837712+0000","last_fullsized":"2024-09-13T13:29:34.837712+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:53.010525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153067+0000","last_change":"2024-09-13T13:29:02.689193+0000","last_active":"2024-09-13T13:29:34.153067+0000","last_peered":"2024-09-13T13:29:34.153067+0000","last_clean":"2024-09-13T13:29:34.153067+0000","last_became_active":"2024-09-13T13:29:02.688387+0000","last_became_peered":"2024-09-13T13:29:02.688387+0000","last_unstale":"2024-09-13T13:29:34.153067+0000","last_undegraded":"2024-09-13T13:29:34.153067+0000","last_fullsized":"2024-09-13T13:29:34.153067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:36.025651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493622+0000","last_change":"2024-09-13T13:29:04.696306+0000","last_active":"2024-09-13T13:29:34.493622+0000","last_peered":"2024-09-13T13:29:34.493622+0000","last_clean":"2024-09-13T13:29:34.493622+0000","last_became_active":"2024-09-13T13:29:04.696029+0000","last_became_peered":"2024-09-13T13:29:04.696029+0000","last_unstale":"2024-09-13T13:29:34.493622+0000","last_undegraded":"2024-09-13T13:29:34.493622+0000","last_fullsized":"2024-09-13T13:29:34.493622+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:59.884068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839467+0000","last_change":"2024-09-13T13:29:06.702539+0000","last_active":"2024-09-13T13:29:34.839467+0000","last_peered":"2024-09-13T13:29:34.839467+0000","last_clean":"2024-09-13T13:29:34.839467+0000","last_became_active":"2024-09-13T13:29:06.702243+0000","last_became_peered":"2024-09-13T13:29:06.702243+0000","last_unstale":"2024-09-13T13:29:34.839467+0000","last_undegraded":"2024-09-13T13:29:34.839467+0000","last_fullsized":"2024-09-13T13:29:34.839467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:31:34.956701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":27,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397516,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1298432,"data_stored":877854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593745,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28272,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716880,"statfs":{"total":95995035648,"available":95966085120,"internally_reserved":0,"allocated":1273856,"data_stored":876333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789975,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713288,"statfs":{"total":95995035648,"available":95962406912,"internally_reserved":0,"allocated":692224,"data_stored":287330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986205,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717424,"statfs":{"total":95995035648,"available":95966642176,"internally_reserved":0,"allocated":716800,"data_stored":287576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27732,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717420,"statfs":{"total":95995035648,"available":95966638080,"internally_reserved":0,"allocated":720896,"data_stored":287292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27688,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717464,"statfs":{"total":95995035648,"available":95966683136,"internally_reserved":0,"allocated":675840,"data_stored":286113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27764,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717388,"statfs":{"total":95995035648,"available":95966605312,"internally_reserved":0,"allocated":753664,"data_stored":288756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738419,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32460,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712692,"statfs":{"total":95995035648,"available":95961796608,"internally_reserved":0,"allocated":1302528,"data_stored":877563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T13:29:47.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T13:29:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.764+0000 7f15f87b5640 1 -- 172.21.15.43:0/2300864145 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f0103b40 msgr2=0x7f15f0103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.764+0000 7f15f87b5640 1 --2- 172.21.15.43:0/2300864145 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f0103b40 0x7f15f0103f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009860 tx=0x7f15ec02a340 comp rx=0 tx=0).stop 2024-09-13T13:29:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.764+0000 7f15f87b5640 1 -- 172.21.15.43:0/2300864145 shutdown_connections 2024-09-13T13:29:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.764+0000 7f15f87b5640 1 --2- 172.21.15.43:0/2300864145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f010d0f0 0x7f15f010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.764+0000 7f15f87b5640 1 --2- 172.21.15.43:0/2300864145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 0x7f15f010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.764+0000 7f15f87b5640 1 --2- 172.21.15.43:0/2300864145 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f0103b40 0x7f15f0103f20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.764+0000 7f15f87b5640 1 -- 172.21.15.43:0/2300864145 >> 172.21.15.43:0/2300864145 conn(0x7f15f00fd710 msgr2=0x7f15f00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 -- 172.21.15.43:0/2300864145 shutdown_connections 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 -- 172.21.15.43:0/2300864145 wait complete. 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 Processor -- start 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 -- start start 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0103b40 0x7f15f01019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 0x7f15f0101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f010d0f0 0x7f15f0102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0112240 con 0x7f15f0104540 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01120c0 con 0x7f15f0103b40 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f87b5640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f15f01123c0 con 0x7f15f010d0f0 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f5d50640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 0x7f15f0101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f6d52640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f010d0f0 0x7f15f0102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f5d50640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 0x7f15f0101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55254/0 (socket says 172.21.15.43:55254) 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f5d50640 1 -- 172.21.15.43:0/994267945 learned_addr learned my addr 172.21.15.43:0/994267945 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.765+0000 7f15f6d52640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f010d0f0 0x7f15f0102440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:51868/0 (socket says 172.21.15.43:51868) 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f6551640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0103b40 0x7f15f01019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f5d50640 1 -- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f010d0f0 msgr2=0x7f15f0102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f5d50640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f010d0f0 0x7f15f0102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f5d50640 1 -- 172.21.15.43:0/994267945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0103b40 msgr2=0x7f15f01019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f5d50640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0103b40 0x7f15f01019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f5d50640 1 -- 172.21.15.43:0/994267945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f01a6c70 con 0x7f15f0104540 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f6d52640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f010d0f0 0x7f15f0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f5d50640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 0x7f15f0101f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15e000e9c0 tx=0x7f15e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15e77fe640 1 -- 172.21.15.43:0/994267945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000cde0 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15e77fe640 1 -- 172.21.15.43:0/994267945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f15e0004470 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f01a6f60 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15e77fe640 1 -- 172.21.15.43:0/994267945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0003e50 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.766+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f01a7460 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.767+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f0103f20 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.768+0000 7f15e77fe640 1 -- 172.21.15.43:0/994267945 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f15e0004000 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.769+0000 7f15e77fe640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f15dc07afb0 0x7f15dc07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.769+0000 7f15f6551640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f15dc07afb0 0x7f15dc07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.769+0000 7f15e77fe640 1 -- 172.21.15.43:0/994267945 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7f15e009e660 con 0x7f15f0104540 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.769+0000 7f15f6551640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f15dc07afb0 0x7f15dc07d470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f15ec02a930 tx=0x7f15ec0058d0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.773+0000 7f15e77fe640 1 -- 172.21.15.43:0/994267945 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f15e0069990 con 0x7f15f0104540 2024-09-13T13:29:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.964+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 --> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15f0106140 con 0x7f15dc07afb0 2024-09-13T13:29:47.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-13T13:29:47.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-13T13:29:47.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:29:47.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:29:47.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[30106]: Deploying daemon alertmanager.a on smithi043 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:29:47.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:29:47.970 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:47 smithi043 ceph-mon[40255]: Deploying daemon alertmanager.a on smithi043 2024-09-13T13:29:47.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.971+0000 7f15e77fe640 1 -- 172.21.15.43:0/994267945 <== mgr.24458 v2:172.21.15.43:6800/3978596566 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346451 (secure 0 0 0) 0x7f15f0106140 con 0x7f15dc07afb0 2024-09-13T13:29:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:29:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-13T13:29:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f15dc07afb0 msgr2=0x7f15dc07d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f15dc07afb0 0x7f15dc07d470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f15ec02a930 tx=0x7f15ec0058d0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 msgr2=0x7f15f0101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 0x7f15f0101f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15e000e9c0 tx=0x7f15e000ee80 comp rx=0 tx=0).stop 2024-09-13T13:29:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 shutdown_connections 2024-09-13T13:29:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f15dc07afb0 0x7f15dc07d470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15f010d0f0 0x7f15f0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f0104540 0x7f15f0101f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 --2- 172.21.15.43:0/994267945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0103b40 0x7f15f01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:47.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.974+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 >> 172.21.15.43:0/994267945 conn(0x7f15f00fd710 msgr2=0x7f15f0104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:47.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.975+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 shutdown_connections 2024-09-13T13:29:47.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:47.975+0000 7f15f87b5640 1 -- 172.21.15.43:0/994267945 wait complete. 2024-09-13T13:29:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-13T13:29:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-13T13:29:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Updating smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T13:29:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:47 smithi189 ceph-mon[32912]: Deploying daemon alertmanager.a on smithi043 2024-09-13T13:29:49.231 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-13T13:29:46.168033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":885,"num_read_kb":742,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":235960,"kb_used_data":7260,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725256,"statfs":{"total":767960285184,"available":767718662144,"internally_reserved":0,"allocated":7434240,"data_stored":4068817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.997157"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837131+0000","last_change":"2024-09-13T13:29:02.691119+0000","last_active":"2024-09-13T13:29:34.837131+0000","last_peered":"2024-09-13T13:29:34.837131+0000","last_clean":"2024-09-13T13:29:34.837131+0000","last_became_active":"2024-09-13T13:29:02.690686+0000","last_became_peered":"2024-09-13T13:29:02.690686+0000","last_unstale":"2024-09-13T13:29:34.837131+0000","last_undegraded":"2024-09-13T13:29:34.837131+0000","last_fullsized":"2024-09-13T13:29:34.837131+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:44:55.882708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836461+0000","last_change":"2024-09-13T13:29:04.698113+0000","last_active":"2024-09-13T13:29:34.836461+0000","last_peered":"2024-09-13T13:29:34.836461+0000","last_clean":"2024-09-13T13:29:34.836461+0000","last_became_active":"2024-09-13T13:29:04.696445+0000","last_became_peered":"2024-09-13T13:29:04.696445+0000","last_unstale":"2024-09-13T13:29:34.836461+0000","last_undegraded":"2024-09-13T13:29:34.836461+0000","last_fullsized":"2024-09-13T13:29:34.836461+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:44.558237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837363+0000","last_change":"2024-09-13T13:29:06.696895+0000","last_active":"2024-09-13T13:29:34.837363+0000","last_peered":"2024-09-13T13:29:34.837363+0000","last_clean":"2024-09-13T13:29:34.837363+0000","last_became_active":"2024-09-13T13:29:06.696575+0000","last_became_peered":"2024-09-13T13:29:06.696575+0000","last_unstale":"2024-09-13T13:29:34.837363+0000","last_undegraded":"2024-09-13T13:29:34.837363+0000","last_fullsized":"2024-09-13T13:29:34.837363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:01.389734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836485+0000","last_change":"2024-09-13T13:29:08.710602+0000","last_active":"2024-09-13T13:29:34.836485+0000","last_peered":"2024-09-13T13:29:34.836485+0000","last_clean":"2024-09-13T13:29:34.836485+0000","last_became_active":"2024-09-13T13:29:08.710327+0000","last_became_peered":"2024-09-13T13:29:08.710327+0000","last_unstale":"2024-09-13T13:29:34.836485+0000","last_undegraded":"2024-09-13T13:29:34.836485+0000","last_fullsized":"2024-09-13T13:29:34.836485+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:03.826279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153202+0000","last_change":"2024-09-13T13:29:08.713800+0000","last_active":"2024-09-13T13:29:34.153202+0000","last_peered":"2024-09-13T13:29:34.153202+0000","last_clean":"2024-09-13T13:29:34.153202+0000","last_became_active":"2024-09-13T13:29:08.713188+0000","last_became_peered":"2024-09-13T13:29:08.713188+0000","last_unstale":"2024-09-13T13:29:34.153202+0000","last_undegraded":"2024-09-13T13:29:34.153202+0000","last_fullsized":"2024-09-13T13:29:34.153202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:38.132777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155384+0000","last_change":"2024-09-13T13:29:02.676910+0000","last_active":"2024-09-13T13:29:34.155384+0000","last_peered":"2024-09-13T13:29:34.155384+0000","last_clean":"2024-09-13T13:29:34.155384+0000","last_became_active":"2024-09-13T13:29:02.676635+0000","last_became_peered":"2024-09-13T13:29:02.676635+0000","last_unstale":"2024-09-13T13:29:34.155384+0000","last_undegraded":"2024-09-13T13:29:34.155384+0000","last_fullsized":"2024-09-13T13:29:34.155384+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:32.568003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153030+0000","last_change":"2024-09-13T13:29:04.690627+0000","last_active":"2024-09-13T13:29:34.153030+0000","last_peered":"2024-09-13T13:29:34.153030+0000","last_clean":"2024-09-13T13:29:34.153030+0000","last_became_active":"2024-09-13T13:29:04.690322+0000","last_became_peered":"2024-09-13T13:29:04.690322+0000","last_unstale":"2024-09-13T13:29:34.153030+0000","last_undegraded":"2024-09-13T13:29:34.153030+0000","last_fullsized":"2024-09-13T13:29:34.153030+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:17:24.264025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836353+0000","last_change":"2024-09-13T13:29:06.697516+0000","last_active":"2024-09-13T13:29:34.836353+0000","last_peered":"2024-09-13T13:29:34.836353+0000","last_clean":"2024-09-13T13:29:34.836353+0000","last_became_active":"2024-09-13T13:29:06.696915+0000","last_became_peered":"2024-09-13T13:29:06.696915+0000","last_unstale":"2024-09-13T13:29:34.836353+0000","last_undegraded":"2024-09-13T13:29:34.836353+0000","last_fullsized":"2024-09-13T13:29:34.836353+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:24.892630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644921+0000","last_change":"2024-09-13T13:29:02.682927+0000","last_active":"2024-09-13T13:29:34.644921+0000","last_peered":"2024-09-13T13:29:34.644921+0000","last_clean":"2024-09-13T13:29:34.644921+0000","last_became_active":"2024-09-13T13:29:02.682512+0000","last_became_peered":"2024-09-13T13:29:02.682512+0000","last_unstale":"2024-09-13T13:29:34.644921+0000","last_undegraded":"2024-09-13T13:29:34.644921+0000","last_fullsized":"2024-09-13T13:29:34.644921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:46:21.323157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836850+0000","last_change":"2024-09-13T13:29:04.687830+0000","last_active":"2024-09-13T13:29:34.836850+0000","last_peered":"2024-09-13T13:29:34.836850+0000","last_clean":"2024-09-13T13:29:34.836850+0000","last_became_active":"2024-09-13T13:29:04.687330+0000","last_became_peered":"2024-09-13T13:29:04.687330+0000","last_unstale":"2024-09-13T13:29:34.836850+0000","last_undegraded":"2024-09-13T13:29:34.836850+0000","last_fullsized":"2024-09-13T13:29:34.836850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:02.875214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644409+0000","last_change":"2024-09-13T13:29:06.702347+0000","last_active":"2024-09-13T13:29:34.644409+0000","last_peered":"2024-09-13T13:29:34.644409+0000","last_clean":"2024-09-13T13:29:34.644409+0000","last_became_active":"2024-09-13T13:29:06.701982+0000","last_became_peered":"2024-09-13T13:29:06.701982+0000","last_unstale":"2024-09-13T13:29:34.644409+0000","last_undegraded":"2024-09-13T13:29:34.644409+0000","last_fullsized":"2024-09-13T13:29:34.644409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:57:11.424454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838568+0000","last_change":"2024-09-13T13:29:08.710877+0000","last_active":"2024-09-13T13:29:34.838568+0000","last_peered":"2024-09-13T13:29:34.838568+0000","last_clean":"2024-09-13T13:29:34.838568+0000","last_became_active":"2024-09-13T13:29:08.710555+0000","last_became_peered":"2024-09-13T13:29:08.710555+0000","last_unstale":"2024-09-13T13:29:34.838568+0000","last_undegraded":"2024-09-13T13:29:34.838568+0000","last_fullsized":"2024-09-13T13:29:34.838568+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:10.659433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644577+0000","last_change":"2024-09-13T13:29:02.683577+0000","last_active":"2024-09-13T13:29:34.644577+0000","last_peered":"2024-09-13T13:29:34.644577+0000","last_clean":"2024-09-13T13:29:34.644577+0000","last_became_active":"2024-09-13T13:29:02.682486+0000","last_became_peered":"2024-09-13T13:29:02.682486+0000","last_unstale":"2024-09-13T13:29:34.644577+0000","last_undegraded":"2024-09-13T13:29:34.644577+0000","last_fullsized":"2024-09-13T13:29:34.644577+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:05.767012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"61'5","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836611+0000","last_change":"2024-09-13T13:29:04.697476+0000","last_active":"2024-09-13T13:29:34.836611+0000","last_peered":"2024-09-13T13:29:34.836611+0000","last_clean":"2024-09-13T13:29:34.836611+0000","last_became_active":"2024-09-13T13:29:04.697175+0000","last_became_peered":"2024-09-13T13:29:04.697175+0000","last_unstale":"2024-09-13T13:29:34.836611+0000","last_undegraded":"2024-09-13T13:29:34.836611+0000","last_fullsized":"2024-09-13T13:29:34.836611+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:43:44.234827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153938+0000","last_change":"2024-09-13T13:29:06.696705+0000","last_active":"2024-09-13T13:29:34.153938+0000","last_peered":"2024-09-13T13:29:34.153938+0000","last_clean":"2024-09-13T13:29:34.153938+0000","last_became_active":"2024-09-13T13:29:06.696422+0000","last_became_peered":"2024-09-13T13:29:06.696422+0000","last_unstale":"2024-09-13T13:29:34.153938+0000","last_undegraded":"2024-09-13T13:29:34.153938+0000","last_fullsized":"2024-09-13T13:29:34.153938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:41.057664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644302+0000","last_change":"2024-09-13T13:29:08.715361+0000","last_active":"2024-09-13T13:29:34.644302+0000","last_peered":"2024-09-13T13:29:34.644302+0000","last_clean":"2024-09-13T13:29:34.644302+0000","last_became_active":"2024-09-13T13:29:08.715023+0000","last_became_peered":"2024-09-13T13:29:08.715023+0000","last_unstale":"2024-09-13T13:29:34.644302+0000","last_undegraded":"2024-09-13T13:29:34.644302+0000","last_fullsized":"2024-09-13T13:29:34.644302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:28:15.702074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836892+0000","last_change":"2024-09-13T13:29:08.704334+0000","last_active":"2024-09-13T13:29:34.836892+0000","last_peered":"2024-09-13T13:29:34.836892+0000","last_clean":"2024-09-13T13:29:34.836892+0000","last_became_active":"2024-09-13T13:29:08.704066+0000","last_became_peered":"2024-09-13T13:29:08.704066+0000","last_unstale":"2024-09-13T13:29:34.836892+0000","last_undegraded":"2024-09-13T13:29:34.836892+0000","last_fullsized":"2024-09-13T13:29:34.836892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:06.053516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.840017+0000","last_change":"2024-09-13T13:29:02.689878+0000","last_active":"2024-09-13T13:29:34.840017+0000","last_peered":"2024-09-13T13:29:34.840017+0000","last_clean":"2024-09-13T13:29:34.840017+0000","last_became_active":"2024-09-13T13:29:02.689376+0000","last_became_peered":"2024-09-13T13:29:02.689376+0000","last_unstale":"2024-09-13T13:29:34.840017+0000","last_undegraded":"2024-09-13T13:29:34.840017+0000","last_fullsized":"2024-09-13T13:29:34.840017+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:34.021660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158393+0000","last_change":"2024-09-13T13:29:04.694389+0000","last_active":"2024-09-13T13:29:34.158393+0000","last_peered":"2024-09-13T13:29:34.158393+0000","last_clean":"2024-09-13T13:29:34.158393+0000","last_became_active":"2024-09-13T13:29:04.694036+0000","last_became_peered":"2024-09-13T13:29:04.694036+0000","last_unstale":"2024-09-13T13:29:34.158393+0000","last_undegraded":"2024-09-13T13:29:34.158393+0000","last_fullsized":"2024-09-13T13:29:34.158393+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:09.377832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841596+0000","last_change":"2024-09-13T13:29:06.705745+0000","last_active":"2024-09-13T13:29:34.841596+0000","last_peered":"2024-09-13T13:29:34.841596+0000","last_clean":"2024-09-13T13:29:34.841596+0000","last_became_active":"2024-09-13T13:29:06.705022+0000","last_became_peered":"2024-09-13T13:29:06.705022+0000","last_unstale":"2024-09-13T13:29:34.841596+0000","last_undegraded":"2024-09-13T13:29:34.841596+0000","last_fullsized":"2024-09-13T13:29:34.841596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:27.783699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155427+0000","last_change":"2024-09-13T13:29:08.711913+0000","last_active":"2024-09-13T13:29:34.155427+0000","last_peered":"2024-09-13T13:29:34.155427+0000","last_clean":"2024-09-13T13:29:34.155427+0000","last_became_active":"2024-09-13T13:29:08.711608+0000","last_became_peered":"2024-09-13T13:29:08.711608+0000","last_unstale":"2024-09-13T13:29:34.155427+0000","last_undegraded":"2024-09-13T13:29:34.155427+0000","last_fullsized":"2024-09-13T13:29:34.155427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:04.109831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.835972+0000","last_change":"2024-09-13T13:29:02.677534+0000","last_active":"2024-09-13T13:29:34.835972+0000","last_peered":"2024-09-13T13:29:34.835972+0000","last_clean":"2024-09-13T13:29:34.835972+0000","last_became_active":"2024-09-13T13:29:02.677185+0000","last_became_peered":"2024-09-13T13:29:02.677185+0000","last_unstale":"2024-09-13T13:29:34.835972+0000","last_undegraded":"2024-09-13T13:29:34.835972+0000","last_fullsized":"2024-09-13T13:29:34.835972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:59.858960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155573+0000","last_change":"2024-09-13T13:29:04.698386+0000","last_active":"2024-09-13T13:29:34.155573+0000","last_peered":"2024-09-13T13:29:34.155573+0000","last_clean":"2024-09-13T13:29:34.155573+0000","last_became_active":"2024-09-13T13:29:04.698086+0000","last_became_peered":"2024-09-13T13:29:04.698086+0000","last_unstale":"2024-09-13T13:29:34.155573+0000","last_undegraded":"2024-09-13T13:29:34.155573+0000","last_fullsized":"2024-09-13T13:29:34.155573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:33.433059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836856+0000","last_change":"2024-09-13T13:29:06.704601+0000","last_active":"2024-09-13T13:29:34.836856+0000","last_peered":"2024-09-13T13:29:34.836856+0000","last_clean":"2024-09-13T13:29:34.836856+0000","last_became_active":"2024-09-13T13:29:06.704322+0000","last_became_peered":"2024-09-13T13:29:06.704322+0000","last_unstale":"2024-09-13T13:29:34.836856+0000","last_undegraded":"2024-09-13T13:29:34.836856+0000","last_fullsized":"2024-09-13T13:29:34.836856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:39.699228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154167+0000","last_change":"2024-09-13T13:29:08.711812+0000","last_active":"2024-09-13T13:29:34.154167+0000","last_peered":"2024-09-13T13:29:34.154167+0000","last_clean":"2024-09-13T13:29:34.154167+0000","last_became_active":"2024-09-13T13:29:08.711491+0000","last_became_peered":"2024-09-13T13:29:08.711491+0000","last_unstale":"2024-09-13T13:29:34.154167+0000","last_undegraded":"2024-09-13T13:29:34.154167+0000","last_fullsized":"2024-09-13T13:29:34.154167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.677336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836598+0000","last_change":"2024-09-13T13:29:02.681192+0000","last_active":"2024-09-13T13:29:34.836598+0000","last_peered":"2024-09-13T13:29:34.836598+0000","last_clean":"2024-09-13T13:29:34.836598+0000","last_became_active":"2024-09-13T13:29:02.680881+0000","last_became_peered":"2024-09-13T13:29:02.680881+0000","last_unstale":"2024-09-13T13:29:34.836598+0000","last_undegraded":"2024-09-13T13:29:34.836598+0000","last_fullsized":"2024-09-13T13:29:34.836598+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:14:08.864821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839546+0000","last_change":"2024-09-13T13:29:04.800696+0000","last_active":"2024-09-13T13:29:34.839546+0000","last_peered":"2024-09-13T13:29:34.839546+0000","last_clean":"2024-09-13T13:29:34.839546+0000","last_became_active":"2024-09-13T13:29:04.800292+0000","last_became_peered":"2024-09-13T13:29:04.800292+0000","last_unstale":"2024-09-13T13:29:34.839546+0000","last_undegraded":"2024-09-13T13:29:34.839546+0000","last_fullsized":"2024-09-13T13:29:34.839546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:43:31.695333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"61'8","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751792+0000","last_change":"2024-09-13T13:29:06.701036+0000","last_active":"2024-09-13T13:29:37.751792+0000","last_peered":"2024-09-13T13:29:37.751792+0000","last_clean":"2024-09-13T13:29:37.751792+0000","last_became_active":"2024-09-13T13:29:06.700680+0000","last_became_peered":"2024-09-13T13:29:06.700680+0000","last_unstale":"2024-09-13T13:29:37.751792+0000","last_undegraded":"2024-09-13T13:29:37.751792+0000","last_fullsized":"2024-09-13T13:29:37.751792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:54.108253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841812+0000","last_change":"2024-09-13T13:29:04.696291+0000","last_active":"2024-09-13T13:29:34.841812+0000","last_peered":"2024-09-13T13:29:34.841812+0000","last_clean":"2024-09-13T13:29:34.841812+0000","last_became_active":"2024-09-13T13:29:04.695972+0000","last_became_peered":"2024-09-13T13:29:04.695972+0000","last_unstale":"2024-09-13T13:29:34.841812+0000","last_undegraded":"2024-09-13T13:29:34.841812+0000","last_fullsized":"2024-09-13T13:29:34.841812+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:41.599634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153155+0000","last_change":"2024-09-13T13:29:02.679531+0000","last_active":"2024-09-13T13:29:34.153155+0000","last_peered":"2024-09-13T13:29:34.153155+0000","last_clean":"2024-09-13T13:29:34.153155+0000","last_became_active":"2024-09-13T13:29:02.679253+0000","last_became_peered":"2024-09-13T13:29:02.679253+0000","last_unstale":"2024-09-13T13:29:34.153155+0000","last_undegraded":"2024-09-13T13:29:34.153155+0000","last_fullsized":"2024-09-13T13:29:34.153155+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:30.644719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751627+0000","last_change":"2024-09-13T13:29:06.701808+0000","last_active":"2024-09-13T13:29:37.751627+0000","last_peered":"2024-09-13T13:29:37.751627+0000","last_clean":"2024-09-13T13:29:37.751627+0000","last_became_active":"2024-09-13T13:29:06.701539+0000","last_became_peered":"2024-09-13T13:29:06.701539+0000","last_unstale":"2024-09-13T13:29:37.751627+0000","last_undegraded":"2024-09-13T13:29:37.751627+0000","last_fullsized":"2024-09-13T13:29:37.751627+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:48:55.894905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644565+0000","last_change":"2024-09-13T13:29:08.710820+0000","last_active":"2024-09-13T13:29:34.644565+0000","last_peered":"2024-09-13T13:29:34.644565+0000","last_clean":"2024-09-13T13:29:34.644565+0000","last_became_active":"2024-09-13T13:29:08.710490+0000","last_became_peered":"2024-09-13T13:29:08.710490+0000","last_unstale":"2024-09-13T13:29:34.644565+0000","last_undegraded":"2024-09-13T13:29:34.644565+0000","last_fullsized":"2024-09-13T13:29:34.644565+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:52.100901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"61'18","reported_seq":58,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152862+0000","last_change":"2024-09-13T13:29:04.801791+0000","last_active":"2024-09-13T13:29:34.152862+0000","last_peered":"2024-09-13T13:29:34.152862+0000","last_clean":"2024-09-13T13:29:34.152862+0000","last_became_active":"2024-09-13T13:29:04.801414+0000","last_became_peered":"2024-09-13T13:29:04.801414+0000","last_unstale":"2024-09-13T13:29:34.152862+0000","last_undegraded":"2024-09-13T13:29:34.152862+0000","last_fullsized":"2024-09-13T13:29:34.152862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:09.169611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155649+0000","last_change":"2024-09-13T13:29:02.690016+0000","last_active":"2024-09-13T13:29:34.155649+0000","last_peered":"2024-09-13T13:29:34.155649+0000","last_clean":"2024-09-13T13:29:34.155649+0000","last_became_active":"2024-09-13T13:29:02.689755+0000","last_became_peered":"2024-09-13T13:29:02.689755+0000","last_unstale":"2024-09-13T13:29:34.155649+0000","last_undegraded":"2024-09-13T13:29:34.155649+0000","last_fullsized":"2024-09-13T13:29:34.155649+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:40.036114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836481+0000","last_change":"2024-09-13T13:29:06.698052+0000","last_active":"2024-09-13T13:29:34.836481+0000","last_peered":"2024-09-13T13:29:34.836481+0000","last_clean":"2024-09-13T13:29:34.836481+0000","last_became_active":"2024-09-13T13:29:06.697745+0000","last_became_peered":"2024-09-13T13:29:06.697745+0000","last_unstale":"2024-09-13T13:29:34.836481+0000","last_undegraded":"2024-09-13T13:29:34.836481+0000","last_fullsized":"2024-09-13T13:29:34.836481+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:49.392569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836639+0000","last_change":"2024-09-13T13:29:08.710852+0000","last_active":"2024-09-13T13:29:34.836639+0000","last_peered":"2024-09-13T13:29:34.836639+0000","last_clean":"2024-09-13T13:29:34.836639+0000","last_became_active":"2024-09-13T13:29:08.709926+0000","last_became_peered":"2024-09-13T13:29:08.709926+0000","last_unstale":"2024-09-13T13:29:34.836639+0000","last_undegraded":"2024-09-13T13:29:34.836639+0000","last_fullsized":"2024-09-13T13:29:34.836639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:56.759617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"61'14","reported_seq":47,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836853+0000","last_change":"2024-09-13T13:29:04.686558+0000","last_active":"2024-09-13T13:29:34.836853+0000","last_peered":"2024-09-13T13:29:34.836853+0000","last_clean":"2024-09-13T13:29:34.836853+0000","last_became_active":"2024-09-13T13:29:04.686224+0000","last_became_peered":"2024-09-13T13:29:04.686224+0000","last_unstale":"2024-09-13T13:29:34.836853+0000","last_undegraded":"2024-09-13T13:29:34.836853+0000","last_fullsized":"2024-09-13T13:29:34.836853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:36:08.808981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839906+0000","last_change":"2024-09-13T13:29:02.692710+0000","last_active":"2024-09-13T13:29:34.839906+0000","last_peered":"2024-09-13T13:29:34.839906+0000","last_clean":"2024-09-13T13:29:34.839906+0000","last_became_active":"2024-09-13T13:29:02.692356+0000","last_became_peered":"2024-09-13T13:29:02.692356+0000","last_unstale":"2024-09-13T13:29:34.839906+0000","last_undegraded":"2024-09-13T13:29:34.839906+0000","last_fullsized":"2024-09-13T13:29:34.839906+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:11.431540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751469+0000","last_change":"2024-09-13T13:29:06.696788+0000","last_active":"2024-09-13T13:29:37.751469+0000","last_peered":"2024-09-13T13:29:37.751469+0000","last_clean":"2024-09-13T13:29:37.751469+0000","last_became_active":"2024-09-13T13:29:06.696486+0000","last_became_peered":"2024-09-13T13:29:06.696486+0000","last_unstale":"2024-09-13T13:29:37.751469+0000","last_undegraded":"2024-09-13T13:29:37.751469+0000","last_fullsized":"2024-09-13T13:29:37.751469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:10.837492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153817+0000","last_change":"2024-09-13T13:29:08.708590+0000","last_active":"2024-09-13T13:29:34.153817+0000","last_peered":"2024-09-13T13:29:34.153817+0000","last_clean":"2024-09-13T13:29:34.153817+0000","last_became_active":"2024-09-13T13:29:08.708295+0000","last_became_peered":"2024-09-13T13:29:08.708295+0000","last_unstale":"2024-09-13T13:29:34.153817+0000","last_undegraded":"2024-09-13T13:29:34.153817+0000","last_fullsized":"2024-09-13T13:29:34.153817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:11.005570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841528+0000","last_change":"2024-09-13T13:29:04.696010+0000","last_active":"2024-09-13T13:29:34.841528+0000","last_peered":"2024-09-13T13:29:34.841528+0000","last_clean":"2024-09-13T13:29:34.841528+0000","last_became_active":"2024-09-13T13:29:04.695623+0000","last_became_peered":"2024-09-13T13:29:04.695623+0000","last_unstale":"2024-09-13T13:29:34.841528+0000","last_undegraded":"2024-09-13T13:29:34.841528+0000","last_fullsized":"2024-09-13T13:29:34.841528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:16.775741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644975+0000","last_change":"2024-09-13T13:29:02.682664+0000","last_active":"2024-09-13T13:29:34.644975+0000","last_peered":"2024-09-13T13:29:34.644975+0000","last_clean":"2024-09-13T13:29:34.644975+0000","last_became_active":"2024-09-13T13:29:02.682242+0000","last_became_peered":"2024-09-13T13:29:02.682242+0000","last_unstale":"2024-09-13T13:29:34.644975+0000","last_undegraded":"2024-09-13T13:29:34.644975+0000","last_fullsized":"2024-09-13T13:29:34.644975+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:07.397695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751466+0000","last_change":"2024-09-13T13:29:06.706857+0000","last_active":"2024-09-13T13:29:37.751466+0000","last_peered":"2024-09-13T13:29:37.751466+0000","last_clean":"2024-09-13T13:29:37.751466+0000","last_became_active":"2024-09-13T13:29:06.706578+0000","last_became_peered":"2024-09-13T13:29:06.706578+0000","last_unstale":"2024-09-13T13:29:37.751466+0000","last_undegraded":"2024-09-13T13:29:37.751466+0000","last_fullsized":"2024-09-13T13:29:37.751466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:47:33.502240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838762+0000","last_change":"2024-09-13T13:29:08.710401+0000","last_active":"2024-09-13T13:29:34.838762+0000","last_peered":"2024-09-13T13:29:34.838762+0000","last_clean":"2024-09-13T13:29:34.838762+0000","last_became_active":"2024-09-13T13:29:08.709855+0000","last_became_peered":"2024-09-13T13:29:08.709855+0000","last_unstale":"2024-09-13T13:29:34.838762+0000","last_undegraded":"2024-09-13T13:29:34.838762+0000","last_fullsized":"2024-09-13T13:29:34.838762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:10.255479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"61'19","reported_seq":62,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.840540+0000","last_change":"2024-09-13T13:29:04.800191+0000","last_active":"2024-09-13T13:29:34.840540+0000","last_peered":"2024-09-13T13:29:34.840540+0000","last_clean":"2024-09-13T13:29:34.840540+0000","last_became_active":"2024-09-13T13:29:04.799486+0000","last_became_peered":"2024-09-13T13:29:04.799486+0000","last_unstale":"2024-09-13T13:29:34.840540+0000","last_undegraded":"2024-09-13T13:29:34.840540+0000","last_fullsized":"2024-09-13T13:29:34.840540+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:44:34.012633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836657+0000","last_change":"2024-09-13T13:29:02.683196+0000","last_active":"2024-09-13T13:29:34.836657+0000","last_peered":"2024-09-13T13:29:34.836657+0000","last_clean":"2024-09-13T13:29:34.836657+0000","last_became_active":"2024-09-13T13:29:02.682794+0000","last_became_peered":"2024-09-13T13:29:02.682794+0000","last_unstale":"2024-09-13T13:29:34.836657+0000","last_undegraded":"2024-09-13T13:29:34.836657+0000","last_fullsized":"2024-09-13T13:29:34.836657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:19:49.873368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493107+0000","last_change":"2024-09-13T13:29:06.701848+0000","last_active":"2024-09-13T13:29:34.493107+0000","last_peered":"2024-09-13T13:29:34.493107+0000","last_clean":"2024-09-13T13:29:34.493107+0000","last_became_active":"2024-09-13T13:29:06.701560+0000","last_became_peered":"2024-09-13T13:29:06.701560+0000","last_unstale":"2024-09-13T13:29:34.493107+0000","last_undegraded":"2024-09-13T13:29:34.493107+0000","last_fullsized":"2024-09-13T13:29:34.493107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:33.137042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837360+0000","last_change":"2024-09-13T13:29:08.709068+0000","last_active":"2024-09-13T13:29:34.837360+0000","last_peered":"2024-09-13T13:29:34.837360+0000","last_clean":"2024-09-13T13:29:34.837360+0000","last_became_active":"2024-09-13T13:29:08.708740+0000","last_became_peered":"2024-09-13T13:29:08.708740+0000","last_unstale":"2024-09-13T13:29:34.837360+0000","last_undegraded":"2024-09-13T13:29:34.837360+0000","last_fullsized":"2024-09-13T13:29:34.837360+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:47.118671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"61'28","reported_seq":83,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751466+0000","last_change":"2024-09-13T13:29:04.696278+0000","last_active":"2024-09-13T13:29:37.751466+0000","last_peered":"2024-09-13T13:29:37.751466+0000","last_clean":"2024-09-13T13:29:37.751466+0000","last_became_active":"2024-09-13T13:29:04.695821+0000","last_became_peered":"2024-09-13T13:29:04.695821+0000","last_unstale":"2024-09-13T13:29:37.751466+0000","last_undegraded":"2024-09-13T13:29:37.751466+0000","last_fullsized":"2024-09-13T13:29:37.751466+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:21:42.012406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.835996+0000","last_change":"2024-09-13T13:29:02.691523+0000","last_active":"2024-09-13T13:29:34.835996+0000","last_peered":"2024-09-13T13:29:34.835996+0000","last_clean":"2024-09-13T13:29:34.835996+0000","last_became_active":"2024-09-13T13:29:02.691189+0000","last_became_peered":"2024-09-13T13:29:02.691189+0000","last_unstale":"2024-09-13T13:29:34.835996+0000","last_undegraded":"2024-09-13T13:29:34.835996+0000","last_fullsized":"2024-09-13T13:29:34.835996+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:28:55.007103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":40,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644495+0000","last_change":"2024-09-13T13:29:04.685741+0000","last_active":"2024-09-13T13:29:34.644495+0000","last_peered":"2024-09-13T13:29:34.644495+0000","last_clean":"2024-09-13T13:29:34.644495+0000","last_became_active":"2024-09-13T13:29:01.674108+0000","last_became_peered":"2024-09-13T13:29:01.674108+0000","last_unstale":"2024-09-13T13:29:34.644495+0000","last_undegraded":"2024-09-13T13:29:34.644495+0000","last_fullsized":"2024-09-13T13:29:34.644495+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_clean_scrub_stamp":"2024-09-13T13:29:00.658664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:21.034317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00141281,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837851+0000","last_change":"2024-09-13T13:29:06.700792+0000","last_active":"2024-09-13T13:29:34.837851+0000","last_peered":"2024-09-13T13:29:34.837851+0000","last_clean":"2024-09-13T13:29:34.837851+0000","last_became_active":"2024-09-13T13:29:06.700296+0000","last_became_peered":"2024-09-13T13:29:06.700296+0000","last_unstale":"2024-09-13T13:29:34.837851+0000","last_undegraded":"2024-09-13T13:29:34.837851+0000","last_fullsized":"2024-09-13T13:29:34.837851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:46.034615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155092+0000","last_change":"2024-09-13T13:29:08.716827+0000","last_active":"2024-09-13T13:29:34.155092+0000","last_peered":"2024-09-13T13:29:34.155092+0000","last_clean":"2024-09-13T13:29:34.155092+0000","last_became_active":"2024-09-13T13:29:08.716184+0000","last_became_peered":"2024-09-13T13:29:08.716184+0000","last_unstale":"2024-09-13T13:29:34.155092+0000","last_undegraded":"2024-09-13T13:29:34.155092+0000","last_fullsized":"2024-09-13T13:29:34.155092+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:52.346008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'13","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841965+0000","last_change":"2024-09-13T13:29:04.686667+0000","last_active":"2024-09-13T13:29:34.841965+0000","last_peered":"2024-09-13T13:29:34.841965+0000","last_clean":"2024-09-13T13:29:34.841965+0000","last_became_active":"2024-09-13T13:29:04.686350+0000","last_became_peered":"2024-09-13T13:29:04.686350+0000","last_unstale":"2024-09-13T13:29:34.841965+0000","last_undegraded":"2024-09-13T13:29:34.841965+0000","last_fullsized":"2024-09-13T13:29:34.841965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:43.433752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836237+0000","last_change":"2024-09-13T13:29:02.681520+0000","last_active":"2024-09-13T13:29:34.836237+0000","last_peered":"2024-09-13T13:29:34.836237+0000","last_clean":"2024-09-13T13:29:34.836237+0000","last_became_active":"2024-09-13T13:29:02.679776+0000","last_became_peered":"2024-09-13T13:29:02.679776+0000","last_unstale":"2024-09-13T13:29:34.836237+0000","last_undegraded":"2024-09-13T13:29:34.836237+0000","last_fullsized":"2024-09-13T13:29:34.836237+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:48.793912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157404+0000","last_change":"2024-09-13T13:29:04.686546+0000","last_active":"2024-09-13T13:29:34.157404+0000","last_peered":"2024-09-13T13:29:34.157404+0000","last_clean":"2024-09-13T13:29:34.157404+0000","last_became_active":"2024-09-13T13:29:01.675417+0000","last_became_peered":"2024-09-13T13:29:01.675417+0000","last_unstale":"2024-09-13T13:29:34.157404+0000","last_undegraded":"2024-09-13T13:29:34.157404+0000","last_fullsized":"2024-09-13T13:29:34.157404+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_clean_scrub_stamp":"2024-09-13T13:29:00.658664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:11.816860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088590200000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158162+0000","last_change":"2024-09-13T13:29:06.698984+0000","last_active":"2024-09-13T13:29:34.158162+0000","last_peered":"2024-09-13T13:29:34.158162+0000","last_clean":"2024-09-13T13:29:34.158162+0000","last_became_active":"2024-09-13T13:29:06.698151+0000","last_became_peered":"2024-09-13T13:29:06.698151+0000","last_unstale":"2024-09-13T13:29:34.158162+0000","last_undegraded":"2024-09-13T13:29:34.158162+0000","last_fullsized":"2024-09-13T13:29:34.158162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:31.909616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154477+0000","last_change":"2024-09-13T13:29:08.718127+0000","last_active":"2024-09-13T13:29:34.154477+0000","last_peered":"2024-09-13T13:29:34.154477+0000","last_clean":"2024-09-13T13:29:34.154477+0000","last_became_active":"2024-09-13T13:29:08.717762+0000","last_became_peered":"2024-09-13T13:29:08.717762+0000","last_unstale":"2024-09-13T13:29:34.154477+0000","last_undegraded":"2024-09-13T13:29:34.154477+0000","last_fullsized":"2024-09-13T13:29:34.154477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:55:32.298438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"61'12","reported_seq":44,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839661+0000","last_change":"2024-09-13T13:29:04.691170+0000","last_active":"2024-09-13T13:29:34.839661+0000","last_peered":"2024-09-13T13:29:34.839661+0000","last_clean":"2024-09-13T13:29:34.839661+0000","last_became_active":"2024-09-13T13:29:04.690864+0000","last_became_peered":"2024-09-13T13:29:04.690864+0000","last_unstale":"2024-09-13T13:29:34.839661+0000","last_undegraded":"2024-09-13T13:29:34.839661+0000","last_fullsized":"2024-09-13T13:29:34.839661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:48.140171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838943+0000","last_change":"2024-09-13T13:29:02.693714+0000","last_active":"2024-09-13T13:29:34.838943+0000","last_peered":"2024-09-13T13:29:34.838943+0000","last_clean":"2024-09-13T13:29:34.838943+0000","last_became_active":"2024-09-13T13:29:02.693382+0000","last_became_peered":"2024-09-13T13:29:02.693382+0000","last_unstale":"2024-09-13T13:29:34.838943+0000","last_undegraded":"2024-09-13T13:29:34.838943+0000","last_fullsized":"2024-09-13T13:29:34.838943+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:39:58.373355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"62'5","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:43.774346+0000","last_change":"2024-09-13T13:29:04.799476+0000","last_active":"2024-09-13T13:29:43.774346+0000","last_peered":"2024-09-13T13:29:43.774346+0000","last_clean":"2024-09-13T13:29:43.774346+0000","last_became_active":"2024-09-13T13:29:01.674373+0000","last_became_peered":"2024-09-13T13:29:01.674373+0000","last_unstale":"2024-09-13T13:29:43.774346+0000","last_undegraded":"2024-09-13T13:29:43.774346+0000","last_fullsized":"2024-09-13T13:29:43.774346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:00.658664+0000","last_clean_scrub_stamp":"2024-09-13T13:29:00.658664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:55.911823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047758099999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644930+0000","last_change":"2024-09-13T13:29:06.703315+0000","last_active":"2024-09-13T13:29:34.644930+0000","last_peered":"2024-09-13T13:29:34.644930+0000","last_clean":"2024-09-13T13:29:34.644930+0000","last_became_active":"2024-09-13T13:29:06.703034+0000","last_became_peered":"2024-09-13T13:29:06.703034+0000","last_unstale":"2024-09-13T13:29:34.644930+0000","last_undegraded":"2024-09-13T13:29:34.644930+0000","last_fullsized":"2024-09-13T13:29:34.644930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:41.495911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841250+0000","last_change":"2024-09-13T13:29:08.715823+0000","last_active":"2024-09-13T13:29:34.841250+0000","last_peered":"2024-09-13T13:29:34.841250+0000","last_clean":"2024-09-13T13:29:34.841250+0000","last_became_active":"2024-09-13T13:29:08.715462+0000","last_became_peered":"2024-09-13T13:29:08.715462+0000","last_unstale":"2024-09-13T13:29:34.841250+0000","last_undegraded":"2024-09-13T13:29:34.841250+0000","last_fullsized":"2024-09-13T13:29:34.841250+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:43:19.382580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'16","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.752075+0000","last_change":"2024-09-13T13:29:04.800994+0000","last_active":"2024-09-13T13:29:37.752075+0000","last_peered":"2024-09-13T13:29:37.752075+0000","last_clean":"2024-09-13T13:29:37.752075+0000","last_became_active":"2024-09-13T13:29:04.800600+0000","last_became_peered":"2024-09-13T13:29:04.800600+0000","last_unstale":"2024-09-13T13:29:37.752075+0000","last_undegraded":"2024-09-13T13:29:37.752075+0000","last_fullsized":"2024-09-13T13:29:37.752075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:32.426534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153677+0000","last_change":"2024-09-13T13:29:02.676473+0000","last_active":"2024-09-13T13:29:34.153677+0000","last_peered":"2024-09-13T13:29:34.153677+0000","last_clean":"2024-09-13T13:29:34.153677+0000","last_became_active":"2024-09-13T13:29:02.676183+0000","last_became_peered":"2024-09-13T13:29:02.676183+0000","last_unstale":"2024-09-13T13:29:34.153677+0000","last_undegraded":"2024-09-13T13:29:34.153677+0000","last_fullsized":"2024-09-13T13:29:34.153677+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:54.804101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"63'79","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:36.623541+0000","last_change":"2024-09-13T13:28:55.532878+0000","last_active":"2024-09-13T13:29:36.623541+0000","last_peered":"2024-09-13T13:29:36.623541+0000","last_clean":"2024-09-13T13:29:36.623541+0000","last_became_active":"2024-09-13T13:28:55.527403+0000","last_became_peered":"2024-09-13T13:28:55.527403+0000","last_unstale":"2024-09-13T13:29:36.623541+0000","last_undegraded":"2024-09-13T13:29:36.623541+0000","last_fullsized":"2024-09-13T13:29:36.623541+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:26:28.508497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:26:28.508497+0000","last_clean_scrub_stamp":"2024-09-13T13:26:28.508497+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:40.153961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154320+0000","last_change":"2024-09-13T13:29:06.698751+0000","last_active":"2024-09-13T13:29:34.154320+0000","last_peered":"2024-09-13T13:29:34.154320+0000","last_clean":"2024-09-13T13:29:34.154320+0000","last_became_active":"2024-09-13T13:29:06.698371+0000","last_became_peered":"2024-09-13T13:29:06.698371+0000","last_unstale":"2024-09-13T13:29:34.154320+0000","last_undegraded":"2024-09-13T13:29:34.154320+0000","last_fullsized":"2024-09-13T13:29:34.154320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:01.079661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644801+0000","last_change":"2024-09-13T13:29:08.715234+0000","last_active":"2024-09-13T13:29:34.644801+0000","last_peered":"2024-09-13T13:29:34.644801+0000","last_clean":"2024-09-13T13:29:34.644801+0000","last_became_active":"2024-09-13T13:29:08.714949+0000","last_became_peered":"2024-09-13T13:29:08.714949+0000","last_unstale":"2024-09-13T13:29:34.644801+0000","last_undegraded":"2024-09-13T13:29:34.644801+0000","last_fullsized":"2024-09-13T13:29:34.644801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:11.765018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837083+0000","last_change":"2024-09-13T13:29:04.696280+0000","last_active":"2024-09-13T13:29:34.837083+0000","last_peered":"2024-09-13T13:29:34.837083+0000","last_clean":"2024-09-13T13:29:34.837083+0000","last_became_active":"2024-09-13T13:29:04.695969+0000","last_became_peered":"2024-09-13T13:29:04.695969+0000","last_unstale":"2024-09-13T13:29:34.837083+0000","last_undegraded":"2024-09-13T13:29:34.837083+0000","last_fullsized":"2024-09-13T13:29:34.837083+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:28:38.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.835958+0000","last_change":"2024-09-13T13:29:02.680615+0000","last_active":"2024-09-13T13:29:34.835958+0000","last_peered":"2024-09-13T13:29:34.835958+0000","last_clean":"2024-09-13T13:29:34.835958+0000","last_became_active":"2024-09-13T13:29:02.680308+0000","last_became_peered":"2024-09-13T13:29:02.680308+0000","last_unstale":"2024-09-13T13:29:34.835958+0000","last_undegraded":"2024-09-13T13:29:34.835958+0000","last_fullsized":"2024-09-13T13:29:34.835958+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:33.074924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644757+0000","last_change":"2024-09-13T13:29:06.697801+0000","last_active":"2024-09-13T13:29:34.644757+0000","last_peered":"2024-09-13T13:29:34.644757+0000","last_clean":"2024-09-13T13:29:34.644757+0000","last_became_active":"2024-09-13T13:29:06.697499+0000","last_became_peered":"2024-09-13T13:29:06.697499+0000","last_unstale":"2024-09-13T13:29:34.644757+0000","last_undegraded":"2024-09-13T13:29:34.644757+0000","last_fullsized":"2024-09-13T13:29:34.644757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:54.456923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153210+0000","last_change":"2024-09-13T13:29:08.707814+0000","last_active":"2024-09-13T13:29:34.153210+0000","last_peered":"2024-09-13T13:29:34.153210+0000","last_clean":"2024-09-13T13:29:34.153210+0000","last_became_active":"2024-09-13T13:29:08.707486+0000","last_became_peered":"2024-09-13T13:29:08.707486+0000","last_unstale":"2024-09-13T13:29:34.153210+0000","last_undegraded":"2024-09-13T13:29:34.153210+0000","last_fullsized":"2024-09-13T13:29:34.153210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:00.243537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"61'17","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837116+0000","last_change":"2024-09-13T13:29:04.698821+0000","last_active":"2024-09-13T13:29:34.837116+0000","last_peered":"2024-09-13T13:29:34.837116+0000","last_clean":"2024-09-13T13:29:34.837116+0000","last_became_active":"2024-09-13T13:29:04.698188+0000","last_became_peered":"2024-09-13T13:29:04.698188+0000","last_unstale":"2024-09-13T13:29:34.837116+0000","last_undegraded":"2024-09-13T13:29:34.837116+0000","last_fullsized":"2024-09-13T13:29:34.837116+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:00.483577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493609+0000","last_change":"2024-09-13T13:29:02.678254+0000","last_active":"2024-09-13T13:29:34.493609+0000","last_peered":"2024-09-13T13:29:34.493609+0000","last_clean":"2024-09-13T13:29:34.493609+0000","last_became_active":"2024-09-13T13:29:02.677913+0000","last_became_peered":"2024-09-13T13:29:02.677913+0000","last_unstale":"2024-09-13T13:29:34.493609+0000","last_undegraded":"2024-09-13T13:29:34.493609+0000","last_fullsized":"2024-09-13T13:29:34.493609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:45.979879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841366+0000","last_change":"2024-09-13T13:29:06.701990+0000","last_active":"2024-09-13T13:29:34.841366+0000","last_peered":"2024-09-13T13:29:34.841366+0000","last_clean":"2024-09-13T13:29:34.841366+0000","last_became_active":"2024-09-13T13:29:06.701684+0000","last_became_peered":"2024-09-13T13:29:06.701684+0000","last_unstale":"2024-09-13T13:29:34.841366+0000","last_undegraded":"2024-09-13T13:29:34.841366+0000","last_fullsized":"2024-09-13T13:29:34.841366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:12.312236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158388+0000","last_change":"2024-09-13T13:29:08.704382+0000","last_active":"2024-09-13T13:29:34.158388+0000","last_peered":"2024-09-13T13:29:34.158388+0000","last_clean":"2024-09-13T13:29:34.158388+0000","last_became_active":"2024-09-13T13:29:08.704028+0000","last_became_peered":"2024-09-13T13:29:08.704028+0000","last_unstale":"2024-09-13T13:29:34.158388+0000","last_undegraded":"2024-09-13T13:29:34.158388+0000","last_fullsized":"2024-09-13T13:29:34.158388+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:02:03.424942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836913+0000","last_change":"2024-09-13T13:29:04.698667+0000","last_active":"2024-09-13T13:29:34.836913+0000","last_peered":"2024-09-13T13:29:34.836913+0000","last_clean":"2024-09-13T13:29:34.836913+0000","last_became_active":"2024-09-13T13:29:04.695505+0000","last_became_peered":"2024-09-13T13:29:04.695505+0000","last_unstale":"2024-09-13T13:29:34.836913+0000","last_undegraded":"2024-09-13T13:29:34.836913+0000","last_fullsized":"2024-09-13T13:29:34.836913+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:13.192158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153532+0000","last_change":"2024-09-13T13:29:02.689313+0000","last_active":"2024-09-13T13:29:34.153532+0000","last_peered":"2024-09-13T13:29:34.153532+0000","last_clean":"2024-09-13T13:29:34.153532+0000","last_became_active":"2024-09-13T13:29:02.688724+0000","last_became_peered":"2024-09-13T13:29:02.688724+0000","last_unstale":"2024-09-13T13:29:34.153532+0000","last_undegraded":"2024-09-13T13:29:34.153532+0000","last_fullsized":"2024-09-13T13:29:34.153532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:31.644591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751480+0000","last_change":"2024-09-13T13:29:06.696961+0000","last_active":"2024-09-13T13:29:37.751480+0000","last_peered":"2024-09-13T13:29:37.751480+0000","last_clean":"2024-09-13T13:29:37.751480+0000","last_became_active":"2024-09-13T13:29:06.696555+0000","last_became_peered":"2024-09-13T13:29:06.696555+0000","last_unstale":"2024-09-13T13:29:37.751480+0000","last_undegraded":"2024-09-13T13:29:37.751480+0000","last_fullsized":"2024-09-13T13:29:37.751480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:53:44.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836846+0000","last_change":"2024-09-13T13:29:08.710508+0000","last_active":"2024-09-13T13:29:34.836846+0000","last_peered":"2024-09-13T13:29:34.836846+0000","last_clean":"2024-09-13T13:29:34.836846+0000","last_became_active":"2024-09-13T13:29:08.710216+0000","last_became_peered":"2024-09-13T13:29:08.710216+0000","last_unstale":"2024-09-13T13:29:34.836846+0000","last_undegraded":"2024-09-13T13:29:34.836846+0000","last_fullsized":"2024-09-13T13:29:34.836846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:56.777900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.840431+0000","last_change":"2024-09-13T13:29:04.692866+0000","last_active":"2024-09-13T13:29:34.840431+0000","last_peered":"2024-09-13T13:29:34.840431+0000","last_clean":"2024-09-13T13:29:34.840431+0000","last_became_active":"2024-09-13T13:29:04.692526+0000","last_became_peered":"2024-09-13T13:29:04.692526+0000","last_unstale":"2024-09-13T13:29:34.840431+0000","last_undegraded":"2024-09-13T13:29:34.840431+0000","last_fullsized":"2024-09-13T13:29:34.840431+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:46.971234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644650+0000","last_change":"2024-09-13T13:29:02.681203+0000","last_active":"2024-09-13T13:29:34.644650+0000","last_peered":"2024-09-13T13:29:34.644650+0000","last_clean":"2024-09-13T13:29:34.644650+0000","last_became_active":"2024-09-13T13:29:02.680424+0000","last_became_peered":"2024-09-13T13:29:02.680424+0000","last_unstale":"2024-09-13T13:29:34.644650+0000","last_undegraded":"2024-09-13T13:29:34.644650+0000","last_fullsized":"2024-09-13T13:29:34.644650+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:01.803414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157811+0000","last_change":"2024-09-13T13:29:06.700622+0000","last_active":"2024-09-13T13:29:34.157811+0000","last_peered":"2024-09-13T13:29:34.157811+0000","last_clean":"2024-09-13T13:29:34.157811+0000","last_became_active":"2024-09-13T13:29:06.700210+0000","last_became_peered":"2024-09-13T13:29:06.700210+0000","last_unstale":"2024-09-13T13:29:34.157811+0000","last_undegraded":"2024-09-13T13:29:34.157811+0000","last_fullsized":"2024-09-13T13:29:34.157811+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:19.443785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839101+0000","last_change":"2024-09-13T13:29:08.712559+0000","last_active":"2024-09-13T13:29:34.839101+0000","last_peered":"2024-09-13T13:29:34.839101+0000","last_clean":"2024-09-13T13:29:34.839101+0000","last_became_active":"2024-09-13T13:29:08.712260+0000","last_became_peered":"2024-09-13T13:29:08.712260+0000","last_unstale":"2024-09-13T13:29:34.839101+0000","last_undegraded":"2024-09-13T13:29:34.839101+0000","last_fullsized":"2024-09-13T13:29:34.839101+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:57.465810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'19","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493264+0000","last_change":"2024-09-13T13:29:04.799952+0000","last_active":"2024-09-13T13:29:34.493264+0000","last_peered":"2024-09-13T13:29:34.493264+0000","last_clean":"2024-09-13T13:29:34.493264+0000","last_became_active":"2024-09-13T13:29:04.799571+0000","last_became_peered":"2024-09-13T13:29:04.799571+0000","last_unstale":"2024-09-13T13:29:34.493264+0000","last_undegraded":"2024-09-13T13:29:34.493264+0000","last_fullsized":"2024-09-13T13:29:34.493264+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:01.616633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154725+0000","last_change":"2024-09-13T13:29:02.680276+0000","last_active":"2024-09-13T13:29:34.154725+0000","last_peered":"2024-09-13T13:29:34.154725+0000","last_clean":"2024-09-13T13:29:34.154725+0000","last_became_active":"2024-09-13T13:29:02.679923+0000","last_became_peered":"2024-09-13T13:29:02.679923+0000","last_unstale":"2024-09-13T13:29:34.154725+0000","last_undegraded":"2024-09-13T13:29:34.154725+0000","last_fullsized":"2024-09-13T13:29:34.154725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:55:02.901598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157564+0000","last_change":"2024-09-13T13:29:06.703939+0000","last_active":"2024-09-13T13:29:34.157564+0000","last_peered":"2024-09-13T13:29:34.157564+0000","last_clean":"2024-09-13T13:29:34.157564+0000","last_became_active":"2024-09-13T13:29:06.702963+0000","last_became_peered":"2024-09-13T13:29:06.702963+0000","last_unstale":"2024-09-13T13:29:34.157564+0000","last_undegraded":"2024-09-13T13:29:34.157564+0000","last_fullsized":"2024-09-13T13:29:34.157564+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:41:40.625430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153914+0000","last_change":"2024-09-13T13:29:08.716797+0000","last_active":"2024-09-13T13:29:34.153914+0000","last_peered":"2024-09-13T13:29:34.153914+0000","last_clean":"2024-09-13T13:29:34.153914+0000","last_became_active":"2024-09-13T13:29:08.716518+0000","last_became_peered":"2024-09-13T13:29:08.716518+0000","last_unstale":"2024-09-13T13:29:34.153914+0000","last_undegraded":"2024-09-13T13:29:34.153914+0000","last_fullsized":"2024-09-13T13:29:34.153914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:12.815349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836648+0000","last_change":"2024-09-13T13:29:04.697964+0000","last_active":"2024-09-13T13:29:34.836648+0000","last_peered":"2024-09-13T13:29:34.836648+0000","last_clean":"2024-09-13T13:29:34.836648+0000","last_became_active":"2024-09-13T13:29:04.697071+0000","last_became_peered":"2024-09-13T13:29:04.697071+0000","last_unstale":"2024-09-13T13:29:34.836648+0000","last_undegraded":"2024-09-13T13:29:34.836648+0000","last_fullsized":"2024-09-13T13:29:34.836648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:03.248740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154769+0000","last_change":"2024-09-13T13:29:02.685296+0000","last_active":"2024-09-13T13:29:34.154769+0000","last_peered":"2024-09-13T13:29:34.154769+0000","last_clean":"2024-09-13T13:29:34.154769+0000","last_became_active":"2024-09-13T13:29:02.684915+0000","last_became_peered":"2024-09-13T13:29:02.684915+0000","last_unstale":"2024-09-13T13:29:34.154769+0000","last_undegraded":"2024-09-13T13:29:34.154769+0000","last_fullsized":"2024-09-13T13:29:34.154769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:35.594765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.157718+0000","last_change":"2024-09-13T13:29:06.699902+0000","last_active":"2024-09-13T13:29:34.157718+0000","last_peered":"2024-09-13T13:29:34.157718+0000","last_clean":"2024-09-13T13:29:34.157718+0000","last_became_active":"2024-09-13T13:29:06.699455+0000","last_became_peered":"2024-09-13T13:29:06.699455+0000","last_unstale":"2024-09-13T13:29:34.157718+0000","last_undegraded":"2024-09-13T13:29:34.157718+0000","last_fullsized":"2024-09-13T13:29:34.157718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:23.533761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153305+0000","last_change":"2024-09-13T13:29:08.714941+0000","last_active":"2024-09-13T13:29:34.153305+0000","last_peered":"2024-09-13T13:29:34.153305+0000","last_clean":"2024-09-13T13:29:34.153305+0000","last_became_active":"2024-09-13T13:29:08.714643+0000","last_became_peered":"2024-09-13T13:29:08.714643+0000","last_unstale":"2024-09-13T13:29:34.153305+0000","last_undegraded":"2024-09-13T13:29:34.153305+0000","last_fullsized":"2024-09-13T13:29:34.153305+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:30:47.476192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644228+0000","last_change":"2024-09-13T13:29:04.799373+0000","last_active":"2024-09-13T13:29:34.644228+0000","last_peered":"2024-09-13T13:29:34.644228+0000","last_clean":"2024-09-13T13:29:34.644228+0000","last_became_active":"2024-09-13T13:29:04.799070+0000","last_became_peered":"2024-09-13T13:29:04.799070+0000","last_unstale":"2024-09-13T13:29:34.644228+0000","last_undegraded":"2024-09-13T13:29:34.644228+0000","last_fullsized":"2024-09-13T13:29:34.644228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:46.513542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"54'2","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154949+0000","last_change":"2024-09-13T13:29:02.689075+0000","last_active":"2024-09-13T13:29:34.154949+0000","last_peered":"2024-09-13T13:29:34.154949+0000","last_clean":"2024-09-13T13:29:34.154949+0000","last_became_active":"2024-09-13T13:29:02.688653+0000","last_became_peered":"2024-09-13T13:29:02.688653+0000","last_unstale":"2024-09-13T13:29:34.154949+0000","last_undegraded":"2024-09-13T13:29:34.154949+0000","last_fullsized":"2024-09-13T13:29:34.154949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:12:07.177401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751790+0000","last_change":"2024-09-13T13:29:06.697615+0000","last_active":"2024-09-13T13:29:37.751790+0000","last_peered":"2024-09-13T13:29:37.751790+0000","last_clean":"2024-09-13T13:29:37.751790+0000","last_became_active":"2024-09-13T13:29:06.697354+0000","last_became_peered":"2024-09-13T13:29:06.697354+0000","last_unstale":"2024-09-13T13:29:37.751790+0000","last_undegraded":"2024-09-13T13:29:37.751790+0000","last_fullsized":"2024-09-13T13:29:37.751790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:46.455383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644284+0000","last_change":"2024-09-13T13:29:08.712561+0000","last_active":"2024-09-13T13:29:34.644284+0000","last_peered":"2024-09-13T13:29:34.644284+0000","last_clean":"2024-09-13T13:29:34.644284+0000","last_became_active":"2024-09-13T13:29:08.712287+0000","last_became_peered":"2024-09-13T13:29:08.712287+0000","last_unstale":"2024-09-13T13:29:34.644284+0000","last_undegraded":"2024-09-13T13:29:34.644284+0000","last_fullsized":"2024-09-13T13:29:34.644284+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:44.552490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493238+0000","last_change":"2024-09-13T13:29:02.688481+0000","last_active":"2024-09-13T13:29:34.493238+0000","last_peered":"2024-09-13T13:29:34.493238+0000","last_clean":"2024-09-13T13:29:34.493238+0000","last_became_active":"2024-09-13T13:29:02.688193+0000","last_became_peered":"2024-09-13T13:29:02.688193+0000","last_unstale":"2024-09-13T13:29:34.493238+0000","last_undegraded":"2024-09-13T13:29:34.493238+0000","last_fullsized":"2024-09-13T13:29:34.493238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:35.246999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"61'6","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155571+0000","last_change":"2024-09-13T13:29:04.800182+0000","last_active":"2024-09-13T13:29:34.155571+0000","last_peered":"2024-09-13T13:29:34.155571+0000","last_clean":"2024-09-13T13:29:34.155571+0000","last_became_active":"2024-09-13T13:29:04.799735+0000","last_became_peered":"2024-09-13T13:29:04.799735+0000","last_unstale":"2024-09-13T13:29:34.155571+0000","last_undegraded":"2024-09-13T13:29:34.155571+0000","last_fullsized":"2024-09-13T13:29:34.155571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:33.977782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644722+0000","last_change":"2024-09-13T13:29:06.699967+0000","last_active":"2024-09-13T13:29:34.644722+0000","last_peered":"2024-09-13T13:29:34.644722+0000","last_clean":"2024-09-13T13:29:34.644722+0000","last_became_active":"2024-09-13T13:29:06.699655+0000","last_became_peered":"2024-09-13T13:29:06.699655+0000","last_unstale":"2024-09-13T13:29:34.644722+0000","last_undegraded":"2024-09-13T13:29:34.644722+0000","last_fullsized":"2024-09-13T13:29:34.644722+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:41.204008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154410+0000","last_change":"2024-09-13T13:29:08.707626+0000","last_active":"2024-09-13T13:29:34.154410+0000","last_peered":"2024-09-13T13:29:34.154410+0000","last_clean":"2024-09-13T13:29:34.154410+0000","last_became_active":"2024-09-13T13:29:08.707305+0000","last_became_peered":"2024-09-13T13:29:08.707305+0000","last_unstale":"2024-09-13T13:29:34.154410+0000","last_undegraded":"2024-09-13T13:29:34.154410+0000","last_fullsized":"2024-09-13T13:29:34.154410+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:25.910973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839793+0000","last_change":"2024-09-13T13:29:04.801476+0000","last_active":"2024-09-13T13:29:34.839793+0000","last_peered":"2024-09-13T13:29:34.839793+0000","last_clean":"2024-09-13T13:29:34.839793+0000","last_became_active":"2024-09-13T13:29:04.801176+0000","last_became_peered":"2024-09-13T13:29:04.801176+0000","last_unstale":"2024-09-13T13:29:34.839793+0000","last_undegraded":"2024-09-13T13:29:34.839793+0000","last_fullsized":"2024-09-13T13:29:34.839793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:12.467846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154049+0000","last_change":"2024-09-13T13:29:02.681133+0000","last_active":"2024-09-13T13:29:34.154049+0000","last_peered":"2024-09-13T13:29:34.154049+0000","last_clean":"2024-09-13T13:29:34.154049+0000","last_became_active":"2024-09-13T13:29:02.680817+0000","last_became_peered":"2024-09-13T13:29:02.680817+0000","last_unstale":"2024-09-13T13:29:34.154049+0000","last_undegraded":"2024-09-13T13:29:34.154049+0000","last_fullsized":"2024-09-13T13:29:34.154049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:18:00.350353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.155469+0000","last_change":"2024-09-13T13:29:06.701850+0000","last_active":"2024-09-13T13:29:34.155469+0000","last_peered":"2024-09-13T13:29:34.155469+0000","last_clean":"2024-09-13T13:29:34.155469+0000","last_became_active":"2024-09-13T13:29:06.701530+0000","last_became_peered":"2024-09-13T13:29:06.701530+0000","last_unstale":"2024-09-13T13:29:34.155469+0000","last_undegraded":"2024-09-13T13:29:34.155469+0000","last_fullsized":"2024-09-13T13:29:34.155469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:34:39.699409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.158045+0000","last_change":"2024-09-13T13:29:08.709476+0000","last_active":"2024-09-13T13:29:34.158045+0000","last_peered":"2024-09-13T13:29:34.158045+0000","last_clean":"2024-09-13T13:29:34.158045+0000","last_became_active":"2024-09-13T13:29:08.709144+0000","last_became_peered":"2024-09-13T13:29:08.709144+0000","last_unstale":"2024-09-13T13:29:34.158045+0000","last_undegraded":"2024-09-13T13:29:34.158045+0000","last_fullsized":"2024-09-13T13:29:34.158045+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:55.180192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644893+0000","last_change":"2024-09-13T13:29:04.801162+0000","last_active":"2024-09-13T13:29:34.644893+0000","last_peered":"2024-09-13T13:29:34.644893+0000","last_clean":"2024-09-13T13:29:34.644893+0000","last_became_active":"2024-09-13T13:29:04.800813+0000","last_became_peered":"2024-09-13T13:29:04.800813+0000","last_unstale":"2024-09-13T13:29:34.644893+0000","last_undegraded":"2024-09-13T13:29:34.644893+0000","last_fullsized":"2024-09-13T13:29:34.644893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:00.728217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837956+0000","last_change":"2024-09-13T13:29:02.689002+0000","last_active":"2024-09-13T13:29:34.837956+0000","last_peered":"2024-09-13T13:29:34.837956+0000","last_clean":"2024-09-13T13:29:34.837956+0000","last_became_active":"2024-09-13T13:29:02.688591+0000","last_became_peered":"2024-09-13T13:29:02.688591+0000","last_unstale":"2024-09-13T13:29:34.837956+0000","last_undegraded":"2024-09-13T13:29:34.837956+0000","last_fullsized":"2024-09-13T13:29:34.837956+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:20.535245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.751731+0000","last_change":"2024-09-13T13:29:06.699492+0000","last_active":"2024-09-13T13:29:37.751731+0000","last_peered":"2024-09-13T13:29:37.751731+0000","last_clean":"2024-09-13T13:29:37.751731+0000","last_became_active":"2024-09-13T13:29:06.699160+0000","last_became_peered":"2024-09-13T13:29:06.699160+0000","last_unstale":"2024-09-13T13:29:37.751731+0000","last_undegraded":"2024-09-13T13:29:37.751731+0000","last_fullsized":"2024-09-13T13:29:37.751731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:37.261283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837130+0000","last_change":"2024-09-13T13:29:08.705412+0000","last_active":"2024-09-13T13:29:34.837130+0000","last_peered":"2024-09-13T13:29:34.837130+0000","last_clean":"2024-09-13T13:29:34.837130+0000","last_became_active":"2024-09-13T13:29:08.705139+0000","last_became_peered":"2024-09-13T13:29:08.705139+0000","last_unstale":"2024-09-13T13:29:34.837130+0000","last_undegraded":"2024-09-13T13:29:34.837130+0000","last_fullsized":"2024-09-13T13:29:34.837130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:01:01.510825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154573+0000","last_change":"2024-09-13T13:29:04.799995+0000","last_active":"2024-09-13T13:29:34.154573+0000","last_peered":"2024-09-13T13:29:34.154573+0000","last_clean":"2024-09-13T13:29:34.154573+0000","last_became_active":"2024-09-13T13:29:04.799606+0000","last_became_peered":"2024-09-13T13:29:04.799606+0000","last_unstale":"2024-09-13T13:29:34.154573+0000","last_undegraded":"2024-09-13T13:29:34.154573+0000","last_fullsized":"2024-09-13T13:29:34.154573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:58.222622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154218+0000","last_change":"2024-09-13T13:29:02.683309+0000","last_active":"2024-09-13T13:29:34.154218+0000","last_peered":"2024-09-13T13:29:34.154218+0000","last_clean":"2024-09-13T13:29:34.154218+0000","last_became_active":"2024-09-13T13:29:02.683000+0000","last_became_peered":"2024-09-13T13:29:02.683000+0000","last_unstale":"2024-09-13T13:29:34.154218+0000","last_undegraded":"2024-09-13T13:29:34.154218+0000","last_fullsized":"2024-09-13T13:29:34.154218+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:42.278169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:37.752109+0000","last_change":"2024-09-13T13:29:06.703646+0000","last_active":"2024-09-13T13:29:37.752109+0000","last_peered":"2024-09-13T13:29:37.752109+0000","last_clean":"2024-09-13T13:29:37.752109+0000","last_became_active":"2024-09-13T13:29:06.703054+0000","last_became_peered":"2024-09-13T13:29:06.703054+0000","last_unstale":"2024-09-13T13:29:37.752109+0000","last_undegraded":"2024-09-13T13:29:37.752109+0000","last_fullsized":"2024-09-13T13:29:37.752109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:26.234583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839327+0000","last_change":"2024-09-13T13:29:08.705234+0000","last_active":"2024-09-13T13:29:34.839327+0000","last_peered":"2024-09-13T13:29:34.839327+0000","last_clean":"2024-09-13T13:29:34.839327+0000","last_became_active":"2024-09-13T13:29:08.704922+0000","last_became_peered":"2024-09-13T13:29:08.704922+0000","last_unstale":"2024-09-13T13:29:34.839327+0000","last_undegraded":"2024-09-13T13:29:34.839327+0000","last_fullsized":"2024-09-13T13:29:34.839327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:53.407453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836675+0000","last_change":"2024-09-13T13:29:04.696565+0000","last_active":"2024-09-13T13:29:34.836675+0000","last_peered":"2024-09-13T13:29:34.836675+0000","last_clean":"2024-09-13T13:29:34.836675+0000","last_became_active":"2024-09-13T13:29:04.696241+0000","last_became_peered":"2024-09-13T13:29:04.696241+0000","last_unstale":"2024-09-13T13:29:34.836675+0000","last_undegraded":"2024-09-13T13:29:34.836675+0000","last_fullsized":"2024-09-13T13:29:34.836675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:50.681411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836007+0000","last_change":"2024-09-13T13:29:02.680918+0000","last_active":"2024-09-13T13:29:34.836007+0000","last_peered":"2024-09-13T13:29:34.836007+0000","last_clean":"2024-09-13T13:29:34.836007+0000","last_became_active":"2024-09-13T13:29:02.680654+0000","last_became_peered":"2024-09-13T13:29:02.680654+0000","last_unstale":"2024-09-13T13:29:34.836007+0000","last_undegraded":"2024-09-13T13:29:34.836007+0000","last_fullsized":"2024-09-13T13:29:34.836007+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:38:55.163478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837677+0000","last_change":"2024-09-13T13:29:06.699233+0000","last_active":"2024-09-13T13:29:34.837677+0000","last_peered":"2024-09-13T13:29:34.837677+0000","last_clean":"2024-09-13T13:29:34.837677+0000","last_became_active":"2024-09-13T13:29:06.698750+0000","last_became_peered":"2024-09-13T13:29:06.698750+0000","last_unstale":"2024-09-13T13:29:34.837677+0000","last_undegraded":"2024-09-13T13:29:34.837677+0000","last_fullsized":"2024-09-13T13:29:34.837677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:18:52.021184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152778+0000","last_change":"2024-09-13T13:29:08.713644+0000","last_active":"2024-09-13T13:29:34.152778+0000","last_peered":"2024-09-13T13:29:34.152778+0000","last_clean":"2024-09-13T13:29:34.152778+0000","last_became_active":"2024-09-13T13:29:08.709970+0000","last_became_peered":"2024-09-13T13:29:08.709970+0000","last_unstale":"2024-09-13T13:29:34.152778+0000","last_undegraded":"2024-09-13T13:29:34.152778+0000","last_fullsized":"2024-09-13T13:29:34.152778+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:13.722611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.841793+0000","last_change":"2024-09-13T13:29:04.694199+0000","last_active":"2024-09-13T13:29:34.841793+0000","last_peered":"2024-09-13T13:29:34.841793+0000","last_clean":"2024-09-13T13:29:34.841793+0000","last_became_active":"2024-09-13T13:29:04.693855+0000","last_became_peered":"2024-09-13T13:29:04.693855+0000","last_unstale":"2024-09-13T13:29:34.841793+0000","last_undegraded":"2024-09-13T13:29:34.841793+0000","last_fullsized":"2024-09-13T13:29:34.841793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:32.409401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.154630+0000","last_change":"2024-09-13T13:29:02.686454+0000","last_active":"2024-09-13T13:29:34.154630+0000","last_peered":"2024-09-13T13:29:34.154630+0000","last_clean":"2024-09-13T13:29:34.154630+0000","last_became_active":"2024-09-13T13:29:02.686126+0000","last_became_peered":"2024-09-13T13:29:02.686126+0000","last_unstale":"2024-09-13T13:29:34.154630+0000","last_undegraded":"2024-09-13T13:29:34.154630+0000","last_fullsized":"2024-09-13T13:29:34.154630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:08.201784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153372+0000","last_change":"2024-09-13T13:29:06.702382+0000","last_active":"2024-09-13T13:29:34.153372+0000","last_peered":"2024-09-13T13:29:34.153372+0000","last_clean":"2024-09-13T13:29:34.153372+0000","last_became_active":"2024-09-13T13:29:06.702042+0000","last_became_peered":"2024-09-13T13:29:06.702042+0000","last_unstale":"2024-09-13T13:29:34.153372+0000","last_undegraded":"2024-09-13T13:29:34.153372+0000","last_fullsized":"2024-09-13T13:29:34.153372+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:23:37.373822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.838435+0000","last_change":"2024-09-13T13:29:08.711167+0000","last_active":"2024-09-13T13:29:34.838435+0000","last_peered":"2024-09-13T13:29:34.838435+0000","last_clean":"2024-09-13T13:29:34.838435+0000","last_became_active":"2024-09-13T13:29:08.710865+0000","last_became_peered":"2024-09-13T13:29:08.710865+0000","last_unstale":"2024-09-13T13:29:34.838435+0000","last_undegraded":"2024-09-13T13:29:34.838435+0000","last_fullsized":"2024-09-13T13:29:34.838435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:32.530162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152879+0000","last_change":"2024-09-13T13:29:04.801077+0000","last_active":"2024-09-13T13:29:34.152879+0000","last_peered":"2024-09-13T13:29:34.152879+0000","last_clean":"2024-09-13T13:29:34.152879+0000","last_became_active":"2024-09-13T13:29:04.800438+0000","last_became_peered":"2024-09-13T13:29:04.800438+0000","last_unstale":"2024-09-13T13:29:34.152879+0000","last_undegraded":"2024-09-13T13:29:34.152879+0000","last_fullsized":"2024-09-13T13:29:34.152879+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:22.074292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.644834+0000","last_change":"2024-09-13T13:29:02.684061+0000","last_active":"2024-09-13T13:29:34.644834+0000","last_peered":"2024-09-13T13:29:34.644834+0000","last_clean":"2024-09-13T13:29:34.644834+0000","last_became_active":"2024-09-13T13:29:02.683750+0000","last_became_peered":"2024-09-13T13:29:02.683750+0000","last_unstale":"2024-09-13T13:29:34.644834+0000","last_undegraded":"2024-09-13T13:29:34.644834+0000","last_fullsized":"2024-09-13T13:29:34.644834+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:19.345241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153649+0000","last_change":"2024-09-13T13:29:06.700724+0000","last_active":"2024-09-13T13:29:34.153649+0000","last_peered":"2024-09-13T13:29:34.153649+0000","last_clean":"2024-09-13T13:29:34.153649+0000","last_became_active":"2024-09-13T13:29:06.700441+0000","last_became_peered":"2024-09-13T13:29:06.700441+0000","last_unstale":"2024-09-13T13:29:34.153649+0000","last_undegraded":"2024-09-13T13:29:34.153649+0000","last_fullsized":"2024-09-13T13:29:34.153649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:45:43.104223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153501+0000","last_change":"2024-09-13T13:29:08.705182+0000","last_active":"2024-09-13T13:29:34.153501+0000","last_peered":"2024-09-13T13:29:34.153501+0000","last_clean":"2024-09-13T13:29:34.153501+0000","last_became_active":"2024-09-13T13:29:08.704848+0000","last_became_peered":"2024-09-13T13:29:08.704848+0000","last_unstale":"2024-09-13T13:29:34.153501+0000","last_undegraded":"2024-09-13T13:29:34.153501+0000","last_fullsized":"2024-09-13T13:29:34.153501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:39.104724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"61'4","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.152753+0000","last_change":"2024-09-13T13:29:04.697197+0000","last_active":"2024-09-13T13:29:34.152753+0000","last_peered":"2024-09-13T13:29:34.152753+0000","last_clean":"2024-09-13T13:29:34.152753+0000","last_became_active":"2024-09-13T13:29:04.696858+0000","last_became_peered":"2024-09-13T13:29:04.696858+0000","last_unstale":"2024-09-13T13:29:34.152753+0000","last_undegraded":"2024-09-13T13:29:34.152753+0000","last_fullsized":"2024-09-13T13:29:34.152753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:49.140079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.836989+0000","last_change":"2024-09-13T13:29:02.691306+0000","last_active":"2024-09-13T13:29:34.836989+0000","last_peered":"2024-09-13T13:29:34.836989+0000","last_clean":"2024-09-13T13:29:34.836989+0000","last_became_active":"2024-09-13T13:29:02.690950+0000","last_became_peered":"2024-09-13T13:29:02.690950+0000","last_unstale":"2024-09-13T13:29:34.836989+0000","last_undegraded":"2024-09-13T13:29:34.836989+0000","last_fullsized":"2024-09-13T13:29:34.836989+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493157+0000","last_change":"2024-09-13T13:29:06.701062+0000","last_active":"2024-09-13T13:29:34.493157+0000","last_peered":"2024-09-13T13:29:34.493157+0000","last_clean":"2024-09-13T13:29:34.493157+0000","last_became_active":"2024-09-13T13:29:06.700697+0000","last_became_peered":"2024-09-13T13:29:06.700697+0000","last_unstale":"2024-09-13T13:29:34.493157+0000","last_undegraded":"2024-09-13T13:29:34.493157+0000","last_fullsized":"2024-09-13T13:29:34.493157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:55.044019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153821+0000","last_change":"2024-09-13T13:29:08.708426+0000","last_active":"2024-09-13T13:29:34.153821+0000","last_peered":"2024-09-13T13:29:34.153821+0000","last_clean":"2024-09-13T13:29:34.153821+0000","last_became_active":"2024-09-13T13:29:08.708161+0000","last_became_peered":"2024-09-13T13:29:08.708161+0000","last_unstale":"2024-09-13T13:29:34.153821+0000","last_undegraded":"2024-09-13T13:29:34.153821+0000","last_fullsized":"2024-09-13T13:29:34.153821+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:01:16.692455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.837712+0000","last_change":"2024-09-13T13:29:08.714777+0000","last_active":"2024-09-13T13:29:34.837712+0000","last_peered":"2024-09-13T13:29:34.837712+0000","last_clean":"2024-09-13T13:29:34.837712+0000","last_became_active":"2024-09-13T13:29:08.714406+0000","last_became_peered":"2024-09-13T13:29:08.714406+0000","last_unstale":"2024-09-13T13:29:34.837712+0000","last_undegraded":"2024-09-13T13:29:34.837712+0000","last_fullsized":"2024-09-13T13:29:34.837712+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:07.685791+0000","last_clean_scrub_stamp":"2024-09-13T13:29:07.685791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:53.010525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.153067+0000","last_change":"2024-09-13T13:29:02.689193+0000","last_active":"2024-09-13T13:29:34.153067+0000","last_peered":"2024-09-13T13:29:34.153067+0000","last_clean":"2024-09-13T13:29:34.153067+0000","last_became_active":"2024-09-13T13:29:02.688387+0000","last_became_peered":"2024-09-13T13:29:02.688387+0000","last_unstale":"2024-09-13T13:29:34.153067+0000","last_undegraded":"2024-09-13T13:29:34.153067+0000","last_fullsized":"2024-09-13T13:29:34.153067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:01.660453+0000","last_clean_scrub_stamp":"2024-09-13T13:29:01.660453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:36.025651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.493622+0000","last_change":"2024-09-13T13:29:04.696306+0000","last_active":"2024-09-13T13:29:34.493622+0000","last_peered":"2024-09-13T13:29:34.493622+0000","last_clean":"2024-09-13T13:29:34.493622+0000","last_became_active":"2024-09-13T13:29:04.696029+0000","last_became_peered":"2024-09-13T13:29:04.696029+0000","last_unstale":"2024-09-13T13:29:34.493622+0000","last_undegraded":"2024-09-13T13:29:34.493622+0000","last_fullsized":"2024-09-13T13:29:34.493622+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:03.672750+0000","last_clean_scrub_stamp":"2024-09-13T13:29:03.672750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:59.884068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T13:29:34.839467+0000","last_change":"2024-09-13T13:29:06.702539+0000","last_active":"2024-09-13T13:29:34.839467+0000","last_peered":"2024-09-13T13:29:34.839467+0000","last_clean":"2024-09-13T13:29:34.839467+0000","last_became_active":"2024-09-13T13:29:06.702243+0000","last_became_peered":"2024-09-13T13:29:06.702243+0000","last_unstale":"2024-09-13T13:29:34.839467+0000","last_undegraded":"2024-09-13T13:29:34.839467+0000","last_fullsized":"2024-09-13T13:29:34.839467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T13:29:05.676464+0000","last_clean_scrub_stamp":"2024-09-13T13:29:05.676464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:31:34.956701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397517,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1298432,"data_stored":877854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593746,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28272,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716880,"statfs":{"total":95995035648,"available":95966085120,"internally_reserved":0,"allocated":1273856,"data_stored":876333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789976,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713288,"statfs":{"total":95995035648,"available":95962406912,"internally_reserved":0,"allocated":692224,"data_stored":287330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986206,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717424,"statfs":{"total":95995035648,"available":95966642176,"internally_reserved":0,"allocated":716800,"data_stored":287576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149733,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27732,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717420,"statfs":{"total":95995035648,"available":95966638080,"internally_reserved":0,"allocated":720896,"data_stored":287292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27688,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717464,"statfs":{"total":95995035648,"available":95966683136,"internally_reserved":0,"allocated":675840,"data_stored":286113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27764,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717388,"statfs":{"total":95995035648,"available":95966605312,"internally_reserved":0,"allocated":753664,"data_stored":288756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738420,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32460,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712692,"statfs":{"total":95995035648,"available":95961796608,"internally_reserved":0,"allocated":1302528,"data_stored":877563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T13:29:49.237 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-13T13:29:49.237 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T13:29:49.237 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-13T13:29:49.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b shell --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-13T13:29:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T13:29:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:49 smithi043 ceph-mon[30106]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:29:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:49 smithi043 ceph-mon[30106]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-13T13:29:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:49 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:29:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:49 smithi043 ceph-mon[40255]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:29:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:49 smithi043 ceph-mon[40255]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-13T13:29:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:49 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:29:50.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 -- 172.21.15.43:0/492765338 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 msgr2=0x7feb74103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:50.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 --2- 172.21.15.43:0/492765338 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74103180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feb70009860 tx=0x7feb7002a340 comp rx=0 tx=0).stop 2024-09-13T13:29:50.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 -- 172.21.15.43:0/492765338 shutdown_connections 2024-09-13T13:29:50.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 --2- 172.21.15.43:0/492765338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74109130 0x7feb7410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 --2- 172.21.15.43:0/492765338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb741087b0 0x7feb74108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 --2- 172.21.15.43:0/492765338 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74103180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 -- 172.21.15.43:0/492765338 >> 172.21.15.43:0/492765338 conn(0x7feb740fe810 msgr2=0x7feb74100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 -- 172.21.15.43:0/492765338 shutdown_connections 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.086+0000 7feb7c49f640 1 -- 172.21.15.43:0/492765338 wait complete. 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 Processor -- start 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 -- start start 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb741087b0 0x7feb74198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb74109130 0x7feb7419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feb74112280 con 0x7feb741087b0 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb74112100 con 0x7feb74109130 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7c49f640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7feb74112400 con 0x7feb74102da0 2024-09-13T13:29:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7a23b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb79a3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb741087b0 0x7feb74198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7a23b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:51900/0 (socket says 172.21.15.43:51900) 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb79a3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb741087b0 0x7feb74198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55274/0 (socket says 172.21.15.43:55274) 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.087+0000 7feb7a23b640 1 -- 172.21.15.43:0/2497878393 learned_addr learned my addr 172.21.15.43:0/2497878393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb7aa3c640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb74109130 0x7feb7419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb79a3a640 1 -- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 msgr2=0x7feb74198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb79a3a640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb79a3a640 1 -- 172.21.15.43:0/2497878393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb74109130 msgr2=0x7feb7419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb79a3a640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb74109130 0x7feb7419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb79a3a640 1 -- 172.21.15.43:0/2497878393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7419e820 con 0x7feb741087b0 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb7a23b640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb7aa3c640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb74109130 0x7feb7419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:29:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb79a3a640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb741087b0 0x7feb74198cb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb6400c970 tx=0x7feb6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb6b7fe640 1 -- 172.21.15.43:0/2497878393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64007bb0 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7419eb10 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb6b7fe640 1 -- 172.21.15.43:0/2497878393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7feb64007d50 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb6b7fe640 1 -- 172.21.15.43:0/2497878393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64018660 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.088+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741a51d0 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.089+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb74103180 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.090+0000 7feb6b7fe640 1 -- 172.21.15.43:0/2497878393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7feb6400f450 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.090+0000 7feb6b7fe640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7feb6007ae10 0x7feb6007d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.091+0000 7feb6b7fe640 1 -- 172.21.15.43:0/2497878393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7421+0+0 (secure 0 0 0) 0x7feb6409e490 con 0x7feb741087b0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.091+0000 7feb7a23b640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7feb6007ae10 0x7feb6007d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.091+0000 7feb7a23b640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7feb6007ae10 0x7feb6007d2d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb700057c0 tx=0x7feb700056d0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:29:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.095+0000 7feb6b7fe640 1 -- 172.21.15.43:0/2497878393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7feb64014030 con 0x7feb741087b0 2024-09-13T13:29:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:49 smithi189 ceph-mon[32912]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T13:29:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:49 smithi189 ceph-mon[32912]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-13T13:29:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:49 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:29:50.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.342+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7feb7410af10 con 0x7feb741087b0 2024-09-13T13:29:50.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.343+0000 7feb6b7fe640 1 -- 172.21.15.43:0/2497878393 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7feb64069740 con 0x7feb741087b0 2024-09-13T13:29:50.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T13:29:50.344 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7feb6007ae10 msgr2=0x7feb6007d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7feb6007ae10 0x7feb6007d2d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb700057c0 tx=0x7feb700056d0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb741087b0 msgr2=0x7feb74198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb741087b0 0x7feb74198cb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb6400c970 tx=0x7feb6400ce30 comp rx=0 tx=0).stop 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 shutdown_connections 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb74109130 0x7feb7419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb741087b0 0x7feb74198cb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7feb6007ae10 0x7feb6007d2d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 --2- 172.21.15.43:0/2497878393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feb74102da0 0x7feb74198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:29:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.345+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 >> 172.21.15.43:0/2497878393 conn(0x7feb740fe810 msgr2=0x7feb74100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:29:50.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.346+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 shutdown_connections 2024-09-13T13:29:50.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T13:29:50.346+0000 7feb7c49f640 1 -- 172.21.15.43:0/2497878393 wait complete. 2024-09-13T13:29:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2497878393' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-13T13:29:50.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2497878393' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-13T13:29:51.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2497878393' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-13T13:29:51.441 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-13T13:29:51.442 INFO:tasks.cephadm:Setup complete, yielding 2024-09-13T13:29:51.442 INFO:teuthology.run_tasks:Running task workunit... 2024-09-13T13:29:51.458 INFO:tasks.workunit:Pulling workunits from ref b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:29:51.459 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-13T13:29:51.459 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-13T13:29:51.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:29:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-13T13:29:51.475 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-13T13:29:51.532 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-13T13:29:51.532 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-13T13:29:51.587 INFO:tasks.workunit:timeout=3h 2024-09-13T13:29:51.587 INFO:tasks.workunit:cleanup=True 2024-09-13T13:29:51.588 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b0201b8c79733293453e7f10a10c7fa43119222b 2024-09-13T13:29:51.644 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-13T13:29:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:51 smithi043 ceph-mon[30106]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-13T13:29:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:51 smithi043 ceph-mon[40255]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-13T13:29:52.069 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:51 smithi189 ceph-mon[32912]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-13T13:29:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:53 smithi043 ceph-mon[30106]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-13T13:29:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:53 smithi043 ceph-mon[40255]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-13T13:29:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:53 smithi189 ceph-mon[32912]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-13T13:29:55.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:29:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:29:55.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:55 smithi043 ceph-mon[30106]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:29:55.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:29:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:29:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:55 smithi043 ceph-mon[40255]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:29:55.993 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:55 smithi043 systemd[1]: Starting Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:29:56.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:55 smithi189 ceph-mon[32912]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:29:56.371 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 podman[74457]: 2024-09-13 13:29:55.973984686 +0000 UTC m=+0.017015757 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-13T13:29:56.371 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 podman[74457]: 2024-09-13 13:29:56.087636617 +0000 UTC m=+0.130667694 volume create 86d305c50070b5089fc7d2a37ed5c29a0364bcdbac0a09b1c461ca463d6aab00 2024-09-13T13:29:56.371 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 podman[74457]: 2024-09-13 13:29:56.17965993 +0000 UTC m=+0.222690957 container create 36cffb406b1fd54f2b1c9f4ae85c3f27322fce151509fc2882baf8d2b50af2e8 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:29:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 podman[74457]: 2024-09-13 13:29:56.482395998 +0000 UTC m=+0.525427027 container init 36cffb406b1fd54f2b1c9f4ae85c3f27322fce151509fc2882baf8d2b50af2e8 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 podman[74457]: 2024-09-13 13:29:56.487844293 +0000 UTC m=+0.530875324 container start 36cffb406b1fd54f2b1c9f4ae85c3f27322fce151509fc2882baf8d2b50af2e8 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.498Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.499Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.499Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.500Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.522Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.522Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.524Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-13T13:29:56.743 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:56.524Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-13T13:29:56.744 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 bash[74457]: 36cffb406b1fd54f2b1c9f4ae85c3f27322fce151509fc2882baf8d2b50af2e8 2024-09-13T13:29:56.744 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:56 smithi043 systemd[1]: Started Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:29:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:29:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:56 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:56 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:56 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:56 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[30106]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[30106]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[40255]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[40255]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:57] ENGINE Bus STOPPING 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:57] ENGINE Bus STOPPED 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:57] ENGINE Bus STARTING 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:57] ENGINE Serving on http://:::9283 2024-09-13T13:29:57.993 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:29:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:29:57] ENGINE Bus STARTED 2024-09-13T13:29:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:57 smithi189 ceph-mon[32912]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:29:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:57 smithi189 ceph-mon[32912]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-13T13:29:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:57 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:57 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:57 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-13T13:29:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:57 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-13T13:29:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[30106]: Deploying daemon grafana.a on smithi189 2024-09-13T13:29:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:29:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:29:58.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:29:58.501Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000284885s 2024-09-13T13:29:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-13T13:29:58.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[40255]: Deploying daemon grafana.a on smithi189 2024-09-13T13:29:58.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:29:58.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:58 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:29:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:58 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-13T13:29:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:58 smithi189 ceph-mon[32912]: Deploying daemon grafana.a on smithi189 2024-09-13T13:29:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:58 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:29:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:58 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:29:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:59 smithi043 ceph-mon[30106]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:29:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:29:59 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:29:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:59 smithi043 ceph-mon[40255]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:29:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:29:59 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:00.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:59 smithi189 ceph-mon[32912]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:00.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:29:59 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:00.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:00 smithi043 ceph-mon[40255]: overall HEALTH_OK 2024-09-13T13:30:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:00 smithi043 ceph-mon[30106]: overall HEALTH_OK 2024-09-13T13:30:01.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:00 smithi189 ceph-mon[32912]: overall HEALTH_OK 2024-09-13T13:30:01.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:01 smithi043 ceph-mon[40255]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:01 smithi043 ceph-mon[30106]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:01 smithi189 ceph-mon[32912]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:03 smithi043 ceph-mon[30106]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:03.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:03 smithi043 ceph-mon[40255]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:03 smithi189 ceph-mon[32912]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:04 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:04 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:05.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:04 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:05.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:30:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:30:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:05 smithi189 ceph-mon[32912]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:05.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:05 smithi043 ceph-mon[30106]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:05 smithi043 ceph-mon[40255]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:05.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:30:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:30:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:06.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:06 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:30:06.504Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003649501s 2024-09-13T13:30:07.090 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:07 smithi043 ceph-mon[30106]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:07 smithi043 ceph-mon[40255]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:08.028 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:07 smithi189 ceph-mon[32912]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:08.028 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:07 smithi189 podman[62454]: 2024-09-13 13:30:07.903302983 +0000 UTC m=+0.119495417 container create c292aacac233f0ff817ecfce2dd3ea69ef73902fe178d1342cea30291c46dfd6 (image=quay.io/ceph/grafana:10.4.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-13T13:30:08.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 podman[62454]: 2024-09-13 13:30:08.155943664 +0000 UTC m=+0.372136100 container init c292aacac233f0ff817ecfce2dd3ea69ef73902fe178d1342cea30291c46dfd6 (image=quay.io/ceph/grafana:10.4.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-13T13:30:08.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 podman[62454]: 2024-09-13 13:30:08.161006608 +0000 UTC m=+0.377199043 container start c292aacac233f0ff817ecfce2dd3ea69ef73902fe178d1342cea30291c46dfd6 (image=quay.io/ceph/grafana:10.4.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 bash[62454]: c292aacac233f0ff817ecfce2dd3ea69ef73902fe178d1342cea30291c46dfd6 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 systemd[1]: Started Ceph grafana.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.285990761Z level=info msg="Starting Grafana" version=10.4.0 commit=03f502a94d17f7dc4e6c34acdf8428aedd986e4c branch=HEAD compiled=2024-09-13T13:30:08Z 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286352239Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286397288Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286463262Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286492334Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286515933Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286540883Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286574975Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-13T13:30:08.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.28660062Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286629603Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286652582Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286675829Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286700622Z level=info msg=Target target=[all] 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286747485Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286771885Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286799059Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286821855Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.28684412Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=settings t=2024-09-13T13:30:08.286869014Z level=info msg="App mode production" 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore t=2024-09-13T13:30:08.287360399Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore t=2024-09-13T13:30:08.287380853Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-13T13:30:08.409 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.28789129Z level=info msg="Starting DB migrations" 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.288775798Z level=info msg="Executing migration" id="create migration_log table" 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.289606141Z level=info msg="Migration successfully executed" id="create migration_log table" duration=829.98µs 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.290091113Z level=info msg="Executing migration" id="create user table" 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.290637017Z level=info msg="Migration successfully executed" id="create user table" duration=545.81µs 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.291069448Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.291603918Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=534.83µs 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.292010415Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.292483358Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=473.041µs 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.292878665Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.293372813Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=494.207µs 2024-09-13T13:30:08.410 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.293814515Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-13T13:30:08.411 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.294282348Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=467.902µs 2024-09-13T13:30:08.411 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.29463256Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.296406962Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=1.773294ms 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.296950236Z level=info msg="Executing migration" id="create user table v2" 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.297527063Z level=info msg="Migration successfully executed" id="create user table v2" duration=578.963µs 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.297881359Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.298363283Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=483.092µs 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.298692624Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.299156072Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=458.872µs 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.299572073Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.299873057Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=301.18µs 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.300302048Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.300704234Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=401.906µs 2024-09-13T13:30:08.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.301068533Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.301811331Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=742.527µs 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.302180394Z level=info msg="Executing migration" id="Update user table charset" 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.302204365Z level=info msg="Migration successfully executed" id="Update user table charset" duration=24.286µs 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.302622882Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.303363825Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=740.145µs 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.303731608Z level=info msg="Executing migration" id="Add missing user data" 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.303886871Z level=info msg="Migration successfully executed" id="Add missing user data" duration=155.344µs 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.30432585Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.305086565Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=760.169µs 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.305460274Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.305944964Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=484.569µs 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.306313292Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.307080439Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=766.651µs 2024-09-13T13:30:08.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.307454141Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.312869078Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=5.411449ms 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.313419987Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.314241377Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=821.34µs 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.314616594Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.314760116Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=143.756µs 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.315165173Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.315637522Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=472.169µs 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.31612814Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.316683036Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=555.385µs 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.317096486Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.317566213Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=469.826µs 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.317915495Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-13T13:30:08.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.318423776Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=507.477µs 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.318806615Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.319276576Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=469.864µs 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.319624042Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.320086359Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=462.114µs 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.320423606Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.320439303Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=16.431µs 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.323349777Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.323835434Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=485.591µs 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.326885775Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.327472811Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=553.817µs 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.327852157Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-13T13:30:08.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.328336076Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=483.832µs 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.328691341Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.329151878Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=460.466µs 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.329549604Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.331643405Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=2.093405ms 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.332436476Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.333002618Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=565.883µs 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.333363786Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.333843837Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=478.054µs 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.334207915Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.334688244Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=480.2µs 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.335203904Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.335680407Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=476.307µs 2024-09-13T13:30:08.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.338577712Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.339066229Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=488.481µs 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.339437495Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.339720423Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=282.87µs 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.342749082Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.34327092Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=526.879µs 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.345637055Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.345966254Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=329.265µs 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.346388841Z level=info msg="Executing migration" id="create star table" 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.346840419Z level=info msg="Migration successfully executed" id="create star table" duration=451.453µs 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.347201163Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.347691357Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=490.035µs 2024-09-13T13:30:08.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.348065179Z level=info msg="Executing migration" id="create org table v1" 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.348568721Z level=info msg="Migration successfully executed" id="create org table v1" duration=504.336µs 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.348930427Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.349420343Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=489.811µs 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.349821175Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.350307275Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=485.9µs 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.35069999Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.351232419Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=532.326µs 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.353742829Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.354380022Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=640.412µs 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.356287262Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.356813677Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=526.296µs 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.359072578Z level=info msg="Executing migration" id="Update org table charset" 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.359105314Z level=info msg="Migration successfully executed" id="Update org table charset" duration=33.347µs 2024-09-13T13:30:08.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.362012071Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.362052804Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=42.566µs 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.364790948Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.365030585Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=241.953µs 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.367419531Z level=info msg="Executing migration" id="create dashboard table" 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.368004324Z level=info msg="Migration successfully executed" id="create dashboard table" duration=584.498µs 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.370187979Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.370749516Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=561.389µs 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.372954223Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.373505493Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=550.938µs 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.375670114Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.376148129Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=477.772µs 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.37829197Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-13T13:30:08.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.378822668Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=530.853µs 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.380796563Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.381297565Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=500.873µs 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.383507647Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.387221878Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.713859ms 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.387612114Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.388161261Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=547.732µs 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.390205589Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.390689554Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=483.84µs 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.393031802Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.3935394Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=507.105µs 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.39565998Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-13T13:30:08.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.395950411Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=290.131µs 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.398361104Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.399128005Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=769.135µs 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.400832288Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.400890829Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=58.838µs 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.403374086Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.404543185Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.168884ms 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.405859376Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.406986975Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.127189ms 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.408619Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-13T13:30:08.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.409781787Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.162878ms 2024-09-13T13:30:08.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.411297138Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-13T13:30:08.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.411789398Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=493.47µs 2024-09-13T13:30:08.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.412156884Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-13T13:30:08.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.413255281Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.098109ms 2024-09-13T13:30:08.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.413606349Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-13T13:30:08.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.414118666Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=510.55µs 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.41447747Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.414959142Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=481.308µs 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.415330516Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.415347149Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=16.34µs 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.41613811Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.416153537Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=16.043µs 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.4165644Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.417728986Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.164373ms 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.418093507Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.419268867Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.175126ms 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.419631119Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-13T13:30:08.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.420782231Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.150789ms 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.421164278Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.422333709Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.169147ms 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.422691579Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.422833136Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=141.524µs 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.423560945Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.424060772Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=499.597µs 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.424444411Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.424911533Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=465.85µs 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.425326564Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.425342832Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=17.012µs 2024-09-13T13:30:08.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.426189387Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.426689652Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=499.864µs 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.427051627Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.427499861Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=448.084µs 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.427887965Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.430970971Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=3.081788ms 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.43132866Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.431778237Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=449.353µs 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.434185154Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.434677708Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=492.461µs 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.437113987Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.437588195Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=475.231µs 2024-09-13T13:30:08.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.439953349Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.440162526Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=209.187µs 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.442698136Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.443082996Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=384.736µs 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.445481955Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.446710705Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.228377ms 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.44833384Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.448815192Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=482.216µs 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.451136001Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.451242959Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=106.997µs 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.454011567Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.454114585Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=103.104µs 2024-09-13T13:30:08.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.45673561Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.457220201Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=484.556µs 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.459521271Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.460777217Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.255699ms 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.462561697Z level=info msg="Executing migration" id="create data_source table" 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.463252302Z level=info msg="Migration successfully executed" id="create data_source table" duration=692.915µs 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.465214288Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.465702953Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=488.388µs 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.468219448Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.468715723Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=495.912µs 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.470731002Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.471216593Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=485.452µs 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.47371935Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-13T13:30:08.664 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.474228711Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=511.757µs 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.476436855Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.479890737Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.454966ms 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.480292651Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.48086557Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=572.766µs 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.483312322Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.483819963Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=507.287µs 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.486116297Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.486730875Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=616.346µs 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.488882901Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.489347545Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=466.934µs 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.491474145Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.49301078Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.538835ms 2024-09-13T13:30:08.665 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.494271314Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.495725427Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.456467ms 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.496815942Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.496849033Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=33.702µs 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.499597166Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.499801321Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=208.877µs 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.502095902Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.503499866Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.403667ms 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.505040549Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.505265776Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=227.735µs 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.507617108Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.507752246Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=133.876µs 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.510313967Z level=info msg="Executing migration" id="Add uid column" 2024-09-13T13:30:08.666 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.511730876Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.416487ms 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.513018888Z level=info msg="Executing migration" id="Update uid value" 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.513144345Z level=info msg="Migration successfully executed" id="Update uid value" duration=125.401µs 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.515857772Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.516429589Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=573.777µs 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.518314666Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.518797206Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=482.325µs 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.52084337Z level=info msg="Executing migration" id="create api_key table" 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.521391889Z level=info msg="Migration successfully executed" id="create api_key table" duration=548.281µs 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.523662554Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-13T13:30:08.667 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.524197442Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=534.628µs 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.526284016Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.526792872Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=511.242µs 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.528745015Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.529300508Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=553.742µs 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.529675977Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.53015683Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=480.719µs 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.530547731Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.531034663Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=486.869µs 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.531364959Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.53183804Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=472.921µs 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.532250146Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.535988306Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=3.737681ms 2024-09-13T13:30:08.668 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.536332151Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.536827855Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=495.427µs 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.537194696Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.537681803Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=486.851µs 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.538015946Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.538491411Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=475.463µs 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.538859936Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.539366161Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=505.92µs 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.539718833Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.539972223Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=253.092µs 2024-09-13T13:30:08.669 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.540579549Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-13T13:30:08.670 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.540955189Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=375.297µs 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.541292529Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.541307499Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=15.746µs 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.542129062Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.543599185Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.469796ms 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.543960568Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.545461415Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.500768ms 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.545812147Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.545921547Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=109.411µs 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.546307703Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.547834288Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.526286ms 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.548214026Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-13T13:30:08.671 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.549757048Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.542675ms 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.550127946Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.550628996Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=501.591µs 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.550985607Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.55136826Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=383.526µs 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.551723105Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.552294728Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=571.455µs 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.552642673Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.553166889Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=523.918µs 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.553538395Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.554083925Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=545.377µs 2024-09-13T13:30:08.672 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.554424734Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.554938636Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=513.646µs 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.555276592Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.555314921Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=38.56µs 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.556114745Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.556129733Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=15.656µs 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.557134956Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.558752341Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.616971ms 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.559120694Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.560740596Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.620403ms 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.561116101Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.561154527Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=38.808µs 2024-09-13T13:30:08.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.561977813Z level=info msg="Executing migration" id="create quota table v1" 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.562433775Z level=info msg="Migration successfully executed" id="create quota table v1" duration=454.689µs 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.562812488Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.563326924Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=514.281µs 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.563687786Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.563702488Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=15.37µs 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.564121112Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.56462576Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=504.336µs 2024-09-13T13:30:08.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.564990727Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.565515949Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=524.986µs 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.565860776Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.567536777Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.675793ms 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.567884758Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.567908701Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=24.486µs 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.56832178Z level=info msg="Executing migration" id="create session table" 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.568879431Z level=info msg="Migration successfully executed" id="create session table" duration=558.597µs 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.571141366Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-13T13:30:08.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.571217696Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=76.39µs 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.574038779Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.574097261Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=58.596µs 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.576907516Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.577493193Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=601.441µs 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.577855807Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.578353305Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=497.302µs 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.580667646Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.58070034Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=33.298µs 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.583531809Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.583546767Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=15.649µs 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.586030705Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-13T13:30:08.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.587917916Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.863363ms 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.588258567Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.590065777Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.806801ms 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.591088755Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.591148962Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=60.308µs 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.594077025Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.594212837Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=138.083µs 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.596956093Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.597460626Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=504.314µs 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.59781244Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.597827859Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=15.358µs 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.60069832Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.602533196Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.834442ms 2024-09-13T13:30:08.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.603546901Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.603657808Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=111.02µs 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.606311667Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.608298029Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.988197ms 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.608651762Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.610481928Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=1.829941ms 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.611654275Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.61175882Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=103.368µs 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.614256262Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.614880742Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=625.728µs 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.616964714Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.617532195Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=568.559µs 2024-09-13T13:30:08.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.617924469Z level=info msg="Executing migration" id="create alert table v1" 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.618679579Z level=info msg="Migration successfully executed" id="create alert table v1" duration=754.873µs 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.62081442Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.621535321Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=722.996µs 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.623414734Z level=info msg="Executing migration" id="add index alert state" 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.624084593Z level=info msg="Migration successfully executed" id="add index alert state" duration=672.118µs 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.625945888Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.62649539Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=549.221µs 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.626851654Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.627322115Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=469.391µs 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.629659246Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.630257653Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=598.07µs 2024-09-13T13:30:08.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.632381576Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.632927705Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=545.888µs 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.634881728Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.640758331Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=5.878551ms 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.641159508Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.641628089Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=468.395µs 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.643834474Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.644382504Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=548.04µs 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.64669698Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-13T13:30:08.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.646906698Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=209.538µs 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.649301567Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.649677039Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=375.335µs 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.650025475Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.650521002Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=495.396µs 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.652662464Z level=info msg="Executing migration" id="Add column is_default" 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.654714545Z level=info msg="Migration successfully executed" id="Add column is_default" duration=2.051829ms 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.655408045Z level=info msg="Executing migration" id="Add column frequency" 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.657623894Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.215613ms 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.658061726Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-13T13:30:08.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.660560186Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.498402ms 2024-09-13T13:30:08.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.660926471Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-13T13:30:08.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.663072387Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.145439ms 2024-09-13T13:30:08.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.663410856Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-13T13:30:08.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.663944976Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=533.781µs 2024-09-13T13:30:08.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.664288435Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-13T13:30:08.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.664304604Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=16.74µs 2024-09-13T13:30:08.910 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.665162171Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-13T13:30:08.910 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.665177597Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=16.143µs 2024-09-13T13:30:08.910 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.665572523Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-13T13:30:08.910 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.666042701Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=469.842µs 2024-09-13T13:30:08.910 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.666413806Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-13T13:30:08.910 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.666941357Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=526.338µs 2024-09-13T13:30:08.910 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.667282742Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.667799618Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=516.626µs 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.668167999Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.668672505Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=504.989µs 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.669021593Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.669543587Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=521.554µs 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.669877251Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.672048097Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.170478ms 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.672386692Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.674602311Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.215343ms 2024-09-13T13:30:08.911 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.674951194Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-13T13:30:08.913 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.675063339Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=112.019µs 2024-09-13T13:30:08.913 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.675439972Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-13T13:30:08.913 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.675962063Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=521.74µs 2024-09-13T13:30:08.913 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.676303716Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-13T13:30:08.913 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.676811985Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=507.106µs 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.677174531Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.679314476Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.139341ms 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.67972724Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.679767717Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=39.082µs 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.68019929Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.680707892Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=507.517µs 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.681044476Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-13T13:30:08.914 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.681623884Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=579.2µs 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.681970199Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.682028696Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=58.65µs 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.682802492Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.683361928Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=559.186µs 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.683713937Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.684235939Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=521.919µs 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.684579166Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.685104505Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=524.828µs 2024-09-13T13:30:08.915 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.685450819Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.68597425Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=523.055µs 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.686318478Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.686891496Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=571.188µs 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.6872481Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.687832885Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=584.572µs 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.68819393Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.688209298Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=16.027µs 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.689101715Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.691656554Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.554484ms 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.692023615Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-13T13:30:08.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.692552925Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=529.272µs 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.69288791Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.695231114Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.342895ms 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.695609741Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.69605453Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=444.733µs 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.696437415Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.697085646Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=647.081µs 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.69755856Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.698348752Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=790.534µs 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.698749821Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-13T13:30:08.917 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.705331515Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=6.579044ms 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.705701853Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.706189995Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=487.898µs 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.706552581Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.707125374Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=572.379µs 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.707511169Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.7077086Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=196.312µs 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.708035732Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.70840646Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=369.706µs 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.708901148Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.709017435Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=116.938µs 2024-09-13T13:30:08.918 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.709750036Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.712101034Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=2.349815ms 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.712476983Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.714785191Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.307875ms 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.715159777Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.715718274Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=558.267µs 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.716085175Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.716654439Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=568.989µs 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.717014419Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.717162005Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=147.444µs 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.718005383Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-13T13:30:08.919 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.721467009Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=3.462723ms 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.721933216Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.722659113Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=723.486µs 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.723115812Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.723233753Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=118.19µs 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.723943934Z level=info msg="Executing migration" id="Move region to single row" 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.724160383Z level=info msg="Migration successfully executed" id="Move region to single row" duration=216.437µs 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.724758375Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.725319271Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=559.658µs 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.725653557Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-13T13:30:08.920 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.726221915Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=567.981µs 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.726580605Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.727149842Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=570.056µs 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.727488681Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.728052061Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=562.849µs 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.728391551Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.728950963Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=559.189µs 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.729298082Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.729832945Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=534.61µs 2024-09-13T13:30:08.921 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.730184356Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.730222959Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=38.991µs 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.731025524Z level=info msg="Executing migration" id="create test_data table" 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.731555288Z level=info msg="Migration successfully executed" id="create test_data table" duration=530.01µs 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.731917155Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.732445402Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=528.743µs 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.732814106Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.733361499Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=547.228µs 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.733711234Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.734316985Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=605.612µs 2024-09-13T13:30:08.922 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.736510947Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.736629113Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=118.408µs 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.739337679Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.739579296Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=241.985µs 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.742249887Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.742314807Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=65.433µs 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.745063426Z level=info msg="Executing migration" id="create team table" 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.745531642Z level=info msg="Migration successfully executed" id="create team table" duration=467.925µs 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.747953928Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.748586368Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=632.383µs 2024-09-13T13:30:08.923 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.75098895Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.751631347Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=643.917µs 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.753712548Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.756218901Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.508633ms 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.756593434Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.756724832Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=132.534µs 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.759553577Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.760209082Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=657.644µs 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.762458878Z level=info msg="Executing migration" id="create team member table" 2024-09-13T13:30:08.924 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.763033317Z level=info msg="Migration successfully executed" id="create team member table" duration=576.149µs 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.765056251Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.765580818Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=524.325µs 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.767893638Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.768442755Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=548.773µs 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.770983516Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.771518908Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=535.157µs 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.7736817Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.776385376Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.703354ms 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.776742191Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.779475487Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=2.732853ms 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.779828792Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.782412762Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=2.583531ms 2024-09-13T13:30:08.925 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.782821721Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.783442053Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=620.067µs 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.786198974Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.786784495Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=583.746µs 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.787135861Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.787759941Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=624.751µs 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.7897944Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.790388785Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=593.985µs 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.792620334Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.793178497Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=557.558µs 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.795314443Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.79587827Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=563.4µs 2024-09-13T13:30:08.926 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.798177471Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-13T13:30:08.927 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.798758332Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=580.536µs 2024-09-13T13:30:08.927 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.79912332Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-13T13:30:08.927 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.799703788Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=580.197µs 2024-09-13T13:30:08.928 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.801990416Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-13T13:30:08.928 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.802507502Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=517.653µs 2024-09-13T13:30:08.928 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.804530939Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-13T13:30:08.928 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.804675196Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=143.202µs 2024-09-13T13:30:08.928 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.807226077Z level=info msg="Executing migration" id="create tag table" 2024-09-13T13:30:08.928 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.807744529Z level=info msg="Migration successfully executed" id="create tag table" duration=518.142µs 2024-09-13T13:30:08.928 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.808098452Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.808649608Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=549.729µs 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.810808057Z level=info msg="Executing migration" id="create login attempt table" 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.811317019Z level=info msg="Migration successfully executed" id="create login attempt table" duration=507.238µs 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.813421735Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.813979863Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=558.119µs 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.816233886Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.816779104Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=545.108µs 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.817119087Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.825206426Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=8.087093ms 2024-09-13T13:30:08.929 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.825567497Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.826035875Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=468.193µs 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.827971527Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.828526605Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=554.997µs 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.830675227Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.830889384Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=214.925µs 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.831227975Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.83161836Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=390.171µs 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.832081078Z level=info msg="Executing migration" id="create user auth table" 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.832530851Z level=info msg="Migration successfully executed" id="create user auth table" duration=449.613µs 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.832933595Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-13T13:30:08.930 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.833502944Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=569.091µs 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.833851015Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.833890083Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=39.369µs 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.834320206Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.837320511Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=3.000258ms 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.837671704Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.840565354Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=2.893272ms 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.840930263Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.844052329Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=3.121725ms 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.844390264Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.847401101Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=3.010667ms 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.847752138Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-13T13:30:08.931 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.848357987Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=605.747µs 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.848721992Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.851566019Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=2.844306ms 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.851930983Z level=info msg="Executing migration" id="create server_lock table" 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.852439605Z level=info msg="Migration successfully executed" id="create server_lock table" duration=508.411µs 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.852808466Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.853381732Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=574.014µs 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.853736935Z level=info msg="Executing migration" id="create user auth token table" 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.854293354Z level=info msg="Migration successfully executed" id="create user auth token table" duration=556.168µs 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.854643041Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.855223101Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=578.697µs 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.855558536Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-13T13:30:08.932 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.856128938Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=570.02µs 2024-09-13T13:30:08.933 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.856465799Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-13T13:30:08.933 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.857103854Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=637.665µs 2024-09-13T13:30:08.933 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.857451379Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-13T13:30:08.934 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.860620184Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.168514ms 2024-09-13T13:30:08.934 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.860975832Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-13T13:30:08.934 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.8615459Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=569.939µs 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.8618888Z level=info msg="Executing migration" id="create cache_data table" 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.862446699Z level=info msg="Migration successfully executed" id="create cache_data table" duration=558.279µs 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.862805375Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.863466535Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=660.919µs 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.863863995Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.864432981Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=568.905µs 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.864797318Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.865420126Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=622.544µs 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.865768936Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.865808546Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=39.065µs 2024-09-13T13:30:08.935 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.866209968Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.866267686Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=57.843µs 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.866642052Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.86720264Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=560.412µs 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.867558951Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.868180859Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=621.641µs 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.868547738Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.869180966Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=633.131µs 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.869530393Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.869568656Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=38.637µs 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.869987664Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-13T13:30:08.936 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.8705663Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=578.557µs 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.870922442Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.871497464Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=574.88µs 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.871840964Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.872469451Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=628.22µs 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.872822172Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.873442443Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=618.971µs 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.873781091Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.877038555Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.255467ms 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.877388975Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.878060252Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=669.374µs 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.878408044Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.878466421Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=58.512µs 2024-09-13T13:30:08.937 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.878854238Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.8794252Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=570.624µs 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.879783492Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.880409836Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=626.155µs 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.880765772Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.881395318Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=629.257µs 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.881744905Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.881782922Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=38.318µs 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.88218688Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.882809192Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=621.98µs 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.883195156Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.883790528Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=594.871µs 2024-09-13T13:30:08.938 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.884165198Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.884780208Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=614.642µs 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.885145171Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.885749658Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=604.348µs 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.886116617Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.889463327Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.346405ms 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.889823268Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.890412267Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=589.038µs 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.890765448Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.891359077Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=593.685µs 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.891713231Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.909355383Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=17.641598ms 2024-09-13T13:30:08.939 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.909794889Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-13T13:30:09.162 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.927053298Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=17.259121ms 2024-09-13T13:30:09.162 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.927436489Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.928069716Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=633.023µs 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.928438353Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.929046398Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=607.763µs 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.92942063Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.932691614Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.270701ms 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.933085378Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.936330912Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.245281ms 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.936783504Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.937468202Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=684.606µs 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.937856959Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.938513157Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=656.125µs 2024-09-13T13:30:09.163 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.938904233Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.93953523Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=630.791µs 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.939915354Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.94060956Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=693.911µs 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.94101031Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.941057557Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=46.941µs 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.941838286Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.945342931Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=3.504471ms 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.94572285Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.949087868Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=3.367262ms 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.949507898Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.952811695Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=3.303637ms 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.953223443Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-13T13:30:09.164 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.953817456Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=593.944µs 2024-09-13T13:30:09.165 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.954188782Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-13T13:30:09.165 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.954827219Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=638.168µs 2024-09-13T13:30:09.165 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.955195692Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-13T13:30:09.165 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.958584906Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=3.388841ms 2024-09-13T13:30:09.165 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.958981578Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-13T13:30:09.165 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.962411947Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.430166ms 2024-09-13T13:30:09.166 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.96280504Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-13T13:30:09.166 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.963458014Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=651.995µs 2024-09-13T13:30:09.166 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.963849773Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-13T13:30:09.166 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.967277213Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.427255ms 2024-09-13T13:30:09.166 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.967662011Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-13T13:30:09.166 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.971075485Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.412997ms 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.971479344Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.971518165Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=39.23µs 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.97198612Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.972731209Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=745.007µs 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.973128857Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.973813893Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=684.77µs 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.974210134Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.974979924Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=752.037µs 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.975364997Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.975404145Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=41.113µs 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.975842628Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.979436488Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.593627ms 2024-09-13T13:30:09.167 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.979819406Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.983387147Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.567532ms 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.983768574Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.987308679Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.539853ms 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.987685999Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.991239269Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.552856ms 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.991622965Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.995131249Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.508ms 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.99554039Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.995579105Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=39.098µs 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.996411574Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.996983822Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=571.936µs 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:08.997357782Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-13T13:30:09.168 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.000960192Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.602299ms 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.001342087Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.001380907Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=39.283µs 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.002208028Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.005864015Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=3.655691ms 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.006263497Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.006880408Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=617.787µs 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.009133862Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.012856547Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.722398ms 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.013254202Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.013773446Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=518.983µs 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.016181696Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-13T13:30:09.169 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.016827749Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=645.932µs 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.018825785Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.022520944Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=3.694855ms 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.022914257Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.023434785Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=519.265µs 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.023858364Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.024501247Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=642.748µs 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.026634802Z level=info msg="Executing migration" id="create alert_image table" 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.027190331Z level=info msg="Migration successfully executed" id="create alert_image table" duration=555.324µs 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.029361491Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.030002238Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=640.444µs 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.03226206Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.032375608Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=114.462µs 2024-09-13T13:30:09.170 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.032878951Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.033499051Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=619.917µs 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.03593397Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.03655115Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=617.114µs 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.038621874Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.038876328Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.041537593Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.041910072Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=376.777µs 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.042248559Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.04289343Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=644.463µs 2024-09-13T13:30:09.171 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.045010442Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.048697838Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=3.687676ms 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.049076733Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.049786184Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=709.617µs 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.052014606Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.052839537Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=828.461µs 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.053232352Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.053786107Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=553.53µs 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.055710548Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.056404966Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=693.075µs 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.058219676Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.058858919Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=638.403µs 2024-09-13T13:30:09.172 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.060826388Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.060844179Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=18.826µs 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.061269949Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.061311359Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=43.84µs 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.06394802Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.064129778Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=181.781µs 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.064539829Z level=info msg="Executing migration" id="create data_keys table" 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.065217817Z level=info msg="Migration successfully executed" id="create data_keys table" duration=677.576µs 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.066793967Z level=info msg="Executing migration" id="create secrets table" 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.067371001Z level=info msg="Migration successfully executed" id="create secrets table" duration=591.369µs 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.069429282Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.090221622Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=20.790585ms 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.090630069Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-13T13:30:09.173 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.094752919Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.122442ms 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.095122436Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.095222143Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=99.549µs 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.097806862Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.118630586Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=20.823565ms 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.119010698Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.140592564Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=21.582032ms 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.1409824Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.141546705Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=564.287µs 2024-09-13T13:30:09.174 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.143807273Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.144582305Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=777.109µs 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.146406226Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.146694269Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=289.906µs 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.148980916Z level=info msg="Executing migration" id="create permission table" 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.149573271Z level=info msg="Migration successfully executed" id="create permission table" duration=594.715µs 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.149955107Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.150577171Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=621.036µs 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.150942526Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.151593775Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=650.887µs 2024-09-13T13:30:09.175 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.151996687Z level=info msg="Executing migration" id="create role table" 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.15256732Z level=info msg="Migration successfully executed" id="create role table" duration=570.44µs 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.152946021Z level=info msg="Executing migration" id="add column display_name" 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.157432265Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.485775ms 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.157797848Z level=info msg="Executing migration" id="add column group_name" 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.162070599Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.272407ms 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.162443641Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.163108129Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=664.154µs 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.163542855Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.164247498Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=704.821µs 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.164599867Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.165295234Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=695.179µs 2024-09-13T13:30:09.176 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.165656136Z level=info msg="Executing migration" id="create team role table" 2024-09-13T13:30:09.412 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-mon[32912]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:09.412 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.412 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.412 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.412 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.412 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-mon[32912]: Deploying daemon node-exporter.a on smithi043 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.166263543Z level=info msg="Migration successfully executed" id="create team role table" duration=607.698µs 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.1666313Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.167310048Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=678.47µs 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.167659823Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.168355268Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=695.11µs 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.168702203Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.169327047Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=624.75µs 2024-09-13T13:30:09.413 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.169662688Z level=info msg="Executing migration" id="create user role table" 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.170203582Z level=info msg="Migration successfully executed" id="create user role table" duration=541.369µs 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.170557553Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.171193972Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=636.244µs 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.171521237Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.172159485Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=636.295µs 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.172492067Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.173113058Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=620.897µs 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.173448119Z level=info msg="Executing migration" id="create builtin role table" 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.173983869Z level=info msg="Migration successfully executed" id="create builtin role table" duration=535.525µs 2024-09-13T13:30:09.414 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.174316992Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.174965873Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=648.657µs 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.175294765Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.17594644Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=651.319µs 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.176278297Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.18094313Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=4.664355ms 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.181281333Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.181932221Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=650.658µs 2024-09-13T13:30:09.415 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.182292374Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.182942851Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=650.146µs 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.183272631Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.18390293Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=621.034µs 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.184226542Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.184847249Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=620.608µs 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.185176375Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.185648963Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=472.347µs 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.186034399Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.186674507Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=639.811µs 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.187006361Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.191671144Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=4.665226ms 2024-09-13T13:30:09.416 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.192025915Z level=info msg="Executing migration" id="permission kind migration" 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.196725136Z level=info msg="Migration successfully executed" id="permission kind migration" duration=4.699095ms 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.197086937Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.201565585Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.478312ms 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.201934122Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.20652592Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=4.591388ms 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.206878932Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.207561807Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=682.644µs 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.207921003Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.208633442Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=712.235µs 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.208988058Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.209624487Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=636.285µs 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.209975033Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-13T13:30:09.417 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.210541727Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=566.642µs 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.210890361Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.211555787Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=665.075µs 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.213789687Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.21384534Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=55.79µs 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.214239827Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.214258521Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=19.38µs 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.216952459Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.217212581Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=263.557µs 2024-09-13T13:30:09.418 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.217591424Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.217909987Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=319.125µs 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.219846473Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.220282415Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=435.823µs 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.222965525Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.223118501Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=153.306µs 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.223543644Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.223825335Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=281.775µs 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.226368682Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.226884952Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=516.131µs 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.229038551Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.229684865Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=643.935µs 2024-09-13T13:30:09.419 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.231781641Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-13T13:30:09.420 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.236427565Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=4.645559ms 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.236771222Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.236809462Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=38.693µs 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.239926423Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.240594614Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=668.026µs 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.242688874Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.243492852Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=806.3µs 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.245314246Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.245960105Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=645.658µs 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.246282301Z level=info msg="Executing migration" id="add correlation config column" 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.251146963Z level=info msg="Migration successfully executed" id="add correlation config column" duration=4.864355ms 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.251524151Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-13T13:30:09.421 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.252191987Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=684.273µs 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.254355012Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.255009224Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=654.097µs 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.257150417Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.270677992Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=13.544279ms 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.271047268Z level=info msg="Executing migration" id="create correlation v2" 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.271757625Z level=info msg="Migration successfully executed" id="create correlation v2" duration=710.014µs 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.273608359Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.274260692Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=653.879µs 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.276328586Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.277103453Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=774.57µs 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.279031555Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.27967187Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=640.424µs 2024-09-13T13:30:09.422 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.280026434Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.280184928Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=158.445µs 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.282771621Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.283335114Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=563.336µs 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.28546044Z level=info msg="Executing migration" id="add provisioning column" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.290292171Z level=info msg="Migration successfully executed" id="add provisioning column" duration=4.831552ms 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.290653694Z level=info msg="Executing migration" id="create entity_events table" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.291197259Z level=info msg="Migration successfully executed" id="create entity_events table" duration=543.43µs 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.291558724Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.292222748Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=663.667µs 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.29433259Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.294645873Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.29712373Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-13T13:30:09.423 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.297405985Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.299724467Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.300414251Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=691.914µs 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.300782897Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.301438898Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=655.706µs 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.303815686Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.304586433Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=772.965µs 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.306626196Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.307374752Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=750.428µs 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.309395716Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.310124972Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=730.477µs 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.310475901Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.311138516Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=662.21µs 2024-09-13T13:30:09.424 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.313246959Z level=info msg="Executing migration" id="Drop public config table" 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.313803261Z level=info msg="Migration successfully executed" id="Drop public config table" duration=556.18µs 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.316047229Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.316727945Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=682.098µs 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.318950098Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.319601741Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=651.367µs 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.31995931Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.320635737Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=676.116µs 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.322698245Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.323514996Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=816.305µs 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.325607531Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.342788439Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=17.180218ms 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.343226465Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-13T13:30:09.425 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.348380972Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=5.15489ms 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.348775904Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.353611748Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=4.833772ms 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.354036187Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.35419233Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=156.688µs 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.357130471Z level=info msg="Executing migration" id="add share column" 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.36206235Z level=info msg="Migration successfully executed" id="add share column" duration=4.932195ms 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.36245834Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.362611198Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=152.958µs 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.365195064Z level=info msg="Executing migration" id="create file table" 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.36585204Z level=info msg="Migration successfully executed" id="create file table" duration=659.66µs 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.367983144Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.368670958Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=687.732µs 2024-09-13T13:30:09.426 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.370595002Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.371272189Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=677.161µs 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.371619095Z level=info msg="Executing migration" id="create file_meta table" 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.372132676Z level=info msg="Migration successfully executed" id="create file_meta table" duration=514.013µs 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.374475639Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.375216095Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=740.351µs 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.37702825Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.377067538Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=39.563µs 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.379732826Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.379811319Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=80.952µs 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.380214341Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.380525172Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=310.732µs 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.382980017Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.383158746Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=180.794µs 2024-09-13T13:30:09.427 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.385676303Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.386498978Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=823.727µs 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.388307145Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.393294673Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=4.987184ms 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.39366261Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.393772162Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=112.649µs 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.396266313Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.397056297Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=789.922µs 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.399052723Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.399385443Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=335.354µs 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.401404746Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.401556188Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=151.917µs 2024-09-13T13:30:09.428 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.404251911Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-13T13:30:09.429 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.404650594Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=400.361µs 2024-09-13T13:30:09.429 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.407004725Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-13T13:30:09.429 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.411841317Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=4.84052ms 2024-09-13T13:30:09.429 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.412238568Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-13T13:30:09.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[40255]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[40255]: Deploying daemon node-exporter.a on smithi043 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[30106]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:09 smithi043 ceph-mon[30106]: Deploying daemon node-exporter.a on smithi043 2024-09-13T13:30:09.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:09 smithi043 bash[74732]: Trying to pull quay.io/prometheus/node-exporter:v1.7.0... 2024-09-13T13:30:09.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.420857063Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=8.618609ms 2024-09-13T13:30:09.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.421234554Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-13T13:30:09.673 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.421927932Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=693.819µs 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.422267446Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.477773216Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=55.50581ms 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.478178978Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.478892886Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=713.562µs 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.479248309Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.479974443Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=726.191µs 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.480333529Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.496631237Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=16.297861ms 2024-09-13T13:30:09.674 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.497075135Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-13T13:30:09.675 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.501990812Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=4.915362ms 2024-09-13T13:30:09.675 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.502349647Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-13T13:30:09.675 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.502539605Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=189.719µs 2024-09-13T13:30:09.675 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.503198942Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-13T13:30:09.675 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.503320536Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=121.746µs 2024-09-13T13:30:09.675 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.504027043Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.504147853Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=120.898µs 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.504522918Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.504638632Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=115.807µs 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.505342316Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.505485984Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=143.742µs 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.505879905Z level=info msg="Executing migration" id="create folder table" 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.506464501Z level=info msg="Migration successfully executed" id="create folder table" duration=583.824µs 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.506814548Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.507628338Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=813.255µs 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.50798656Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-13T13:30:09.676 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.508696055Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=711.04µs 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.509070361Z level=info msg="Executing migration" id="Update folder title length" 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.509086346Z level=info msg="Migration successfully executed" id="Update folder title length" duration=16.733µs 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.509953823Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.510661615Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=707.458µs 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.511008648Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.511657502Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=648.732µs 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.511992065Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.512708946Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=716.52µs 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.51307048Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.513361765Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=291.191µs 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.513951128Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.51412723Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=175.983µs 2024-09-13T13:30:09.677 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.514957266Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.515689552Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=734.475µs 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.516022522Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.516719463Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=695.467µs 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.517060417Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.517708315Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=647.651µs 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.518036748Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.518750996Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=713.907µs 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.519116088Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.519762436Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=646.197µs 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.520111199Z level=info msg="Executing migration" id="create anon_device table" 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.52066761Z level=info msg="Migration successfully executed" id="create anon_device table" duration=556.211µs 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.521034328Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-13T13:30:09.678 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.521781898Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=747.368µs 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.522209557Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.5229385Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=728.699µs 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.52333873Z level=info msg="Executing migration" id="create signing_key table" 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.524007583Z level=info msg="Migration successfully executed" id="create signing_key table" duration=668.578µs 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.524363194Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.525039383Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=676.04µs 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.525377379Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.526070786Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=693.173µs 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.526397615Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.526561375Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=165.428µs 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.527372095Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.533103519Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=5.733385ms 2024-09-13T13:30:09.679 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.533498145Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-13T13:30:09.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.533981639Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=484.005µs 2024-09-13T13:30:09.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.534415204Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-13T13:30:09.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.535121328Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=705.767µs 2024-09-13T13:30:09.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.537344689Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-13T13:30:09.680 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.538039222Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=694.442µs 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.540232513Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.540916608Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=683.881µs 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.542987227Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.543706479Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=719.014µs 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.544093787Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.544786441Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=691.62µs 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.54702262Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.547760712Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=740.329µs 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.549567366Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.550048716Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=481.939µs 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.552496145Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.552763307Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=270.106µs 2024-09-13T13:30:09.681 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.55509006Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.555137846Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=48.447µs 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.555579357Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.560717839Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=5.138036ms 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.561079263Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.566375912Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.296452ms 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.566727984Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.56694423Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=216.253µs 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=migrator t=2024-09-13T13:30:09.567313947Z level=info msg="migrations completed" performed=547 skipped=0 duration=1.278571612s 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore t=2024-09-13T13:30:09.568159052Z level=info msg="Created default organization" 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=secrets t=2024-09-13T13:30:09.568663253Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=plugin.store t=2024-09-13T13:30:09.583171431Z level=info msg="Loading plugins..." 2024-09-13T13:30:09.682 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=local.finder t=2024-09-13T13:30:09.626328164Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=plugin.store t=2024-09-13T13:30:09.626346696Z level=info msg="Plugins loaded" count=55 duration=43.191885ms 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=query_data t=2024-09-13T13:30:09.628407236Z level=info msg="Query Service initialization" 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=live.push_http t=2024-09-13T13:30:09.630760951Z level=info msg="Live Push Gateway initialization" 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.migration t=2024-09-13T13:30:09.631833052Z level=info msg=Starting 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.migration t=2024-09-13T13:30:09.632109074Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.migration orgID=1 t=2024-09-13T13:30:09.632368566Z level=info msg="Migrating alerts for organisation" 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.migration orgID=1 t=2024-09-13T13:30:09.632790284Z level=info msg="Alerts found to migrate" alerts=0 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.migration t=2024-09-13T13:30:09.633888967Z level=info msg="Completed alerting migration" 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.state.manager t=2024-09-13T13:30:09.644761847Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-13T13:30:09.683 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=infra.usagestats.collector t=2024-09-13T13:30:09.646104649Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=provisioning.datasources t=2024-09-13T13:30:09.647681509Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=provisioning.datasources t=2024-09-13T13:30:09.660278639Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=provisioning.alerting t=2024-09-13T13:30:09.672795933Z level=info msg="starting to provision alerting" 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=provisioning.alerting t=2024-09-13T13:30:09.672807308Z level=info msg="finished to provision alerting" 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=grafanaStorageLogger t=2024-09-13T13:30:09.672964317Z level=info msg="Storage starting" 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.state.manager t=2024-09-13T13:30:09.673944428Z level=info msg="Warming state cache for startup" 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.multiorg.alertmanager t=2024-09-13T13:30:09.674756943Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=http.server t=2024-09-13T13:30:09.676573001Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-13T13:30:09.684 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=http.server t=2024-09-13T13:30:09.676890382Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-13T13:30:09.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:09 smithi043 bash[74732]: Getting image source signatures 2024-09-13T13:30:09.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:09 smithi043 bash[74732]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-13T13:30:09.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:09 smithi043 bash[74732]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-13T13:30:09.992 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:09 smithi043 bash[74732]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-13T13:30:10.148 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore.transactions t=2024-09-13T13:30:09.705005156Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-13T13:30:10.148 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.state.manager t=2024-09-13T13:30:09.720426922Z level=info msg="State cache has been initialized" states=0 duration=46.481944ms 2024-09-13T13:30:10.148 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ngalert.scheduler t=2024-09-13T13:30:09.720447268Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ticker t=2024-09-13T13:30:09.720482471Z level=info msg=starting first_tick=2024-09-13T13:30:10Z 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=provisioning.dashboard t=2024-09-13T13:30:09.734779204Z level=info msg="starting to provision dashboards" 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore.transactions t=2024-09-13T13:30:09.747221577Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=grafana.update.checker t=2024-09-13T13:30:09.753977457Z level=info msg="Update check succeeded" duration=79.147096ms 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=plugins.update.checker t=2024-09-13T13:30:09.754369684Z level=info msg="Update check succeeded" duration=80.798719ms 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore.transactions t=2024-09-13T13:30:09.758062254Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore.transactions t=2024-09-13T13:30:09.767953385Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore.transactions t=2024-09-13T13:30:09.783693249Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-13T13:30:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=provisioning.dashboard t=2024-09-13T13:30:09.887405742Z level=info msg="finished to provision dashboards" 2024-09-13T13:30:10.633 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:10 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=grafana-apiserver t=2024-09-13T13:30:10.157276801Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-13T13:30:10.633 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:30:10 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=grafana-apiserver t=2024-09-13T13:30:10.15770429Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-13T13:30:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:10 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:10 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:10 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:11.491 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:11 smithi043 bash[74732]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-13T13:30:11.491 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:11 smithi043 bash[74732]: Writing manifest to image destination 2024-09-13T13:30:11.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:11 smithi043 ceph-mon[30106]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:11.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:11 smithi043 ceph-mon[40255]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:11.884 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:11 smithi043 podman[74732]: 2024-09-13 13:30:11.640071875 +0000 UTC m=+2.413302360 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-13T13:30:11.884 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:11 smithi043 podman[74732]: 2024-09-13 13:30:11.755678376 +0000 UTC m=+2.528908860 container create 458e3f697bcfe2d59dac647f71c370d835686a43f06eaf541313a440b5a2791b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:11 smithi189 ceph-mon[32912]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:12.492 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:11 smithi043 podman[74732]: 2024-09-13 13:30:11.999374758 +0000 UTC m=+2.772605272 container init 458e3f697bcfe2d59dac647f71c370d835686a43f06eaf541313a440b5a2791b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:12.492 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 podman[74732]: 2024-09-13 13:30:12.00427636 +0000 UTC m=+2.777506847 container start 458e3f697bcfe2d59dac647f71c370d835686a43f06eaf541313a440b5a2791b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:12.492 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.008Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-13T13:30:12.492 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.008Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-13T13:30:12.492 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.008Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-13T13:30:12.492 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.008Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-13T13:30:12.492 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.008Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=arp 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-13T13:30:12.493 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-13T13:30:12.494 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-13T13:30:12.494 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-13T13:30:12.494 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=edac 2024-09-13T13:30:12.494 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-13T13:30:12.494 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-13T13:30:12.495 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=os 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-13T13:30:12.496 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=stat 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.009Z caller=node_exporter.go:117 level=info collector=time 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.010Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.010Z caller=node_exporter.go:117 level=info collector=uname 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.010Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.010Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.010Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.010Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a[74808]: ts=2024-09-13T13:30:12.010Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-13T13:30:12.497 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 bash[74732]: 458e3f697bcfe2d59dac647f71c370d835686a43f06eaf541313a440b5a2791b 2024-09-13T13:30:12.498 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:30:12 smithi043 systemd[1]: Started Ceph node-exporter.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:30:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:13 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:13 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:13 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:13 smithi189 ceph-mon[32912]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:13 smithi189 ceph-mon[32912]: Deploying daemon node-exporter.b on smithi189 2024-09-13T13:30:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[30106]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[30106]: Deploying daemon node-exporter.b on smithi189 2024-09-13T13:30:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[40255]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:13 smithi043 ceph-mon[40255]: Deploying daemon node-exporter.b on smithi189 2024-09-13T13:30:13.898 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:13 smithi189 bash[62709]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-13T13:30:15.148 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:14 smithi189 bash[62709]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-13T13:30:15.148 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 bash[62709]: Writing manifest to image destination 2024-09-13T13:30:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:15 smithi043 ceph-mon[30106]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:15 smithi043 ceph-mon[40255]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:15.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-mon[32912]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:15.517 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 podman[62709]: 2024-09-13 13:30:15.302368658 +0000 UTC m=+2.286816395 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-13T13:30:15.517 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 podman[62709]: 2024-09-13 13:30:15.392541049 +0000 UTC m=+2.376988772 container create 22f641d70cf75cfbc2503bbc5042f223c78eb14d87633197d7e01493eeb07d8f (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-13T13:30:15.517 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 podman[62709]: 2024-09-13 13:30:15.628431494 +0000 UTC m=+2.612879224 container init 22f641d70cf75cfbc2503bbc5042f223c78eb14d87633197d7e01493eeb07d8f (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 podman[62709]: 2024-09-13 13:30:15.633623218 +0000 UTC m=+2.618070942 container start 22f641d70cf75cfbc2503bbc5042f223c78eb14d87633197d7e01493eeb07d8f (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.636Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.636Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.637Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.637Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.637Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.637Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-13T13:30:15.900 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.637Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.637Z caller=node_exporter.go:117 level=info collector=arp 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=edac 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-13T13:30:15.901 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-13T13:30:15.902 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=os 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=stat 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-13T13:30:15.903 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=time 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=uname 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b[62784]: ts=2024-09-13T13:30:15.638Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 bash[62709]: 22f641d70cf75cfbc2503bbc5042f223c78eb14d87633197d7e01493eeb07d8f 2024-09-13T13:30:15.904 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:30:15 smithi189 systemd[1]: Started Ceph node-exporter.b for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:30:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:30:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:30:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:16.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:16 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:30:16.585 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:16.585 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:16 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.585 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:16 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.585 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:16 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.585 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:16 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.586 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:16 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:16.586 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:16 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:30:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:17 smithi043 ceph-mon[30106]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:17 smithi043 ceph-mon[40255]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:17.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:17 smithi189 ceph-mon[32912]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:19 smithi189 ceph-mon[32912]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:19.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:19 smithi043 ceph-mon[30106]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:19.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:19 smithi043 ceph-mon[40255]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:30:20.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:30:20.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:30:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:30:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:21 smithi189 ceph-mon[32912]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:21 smithi189 ceph-mon[32912]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-13T13:30:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:21 smithi189 ceph-mon[32912]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-13T13:30:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:21 smithi043 ceph-mon[30106]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:21 smithi043 ceph-mon[30106]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-13T13:30:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:21 smithi043 ceph-mon[30106]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-13T13:30:21.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:21 smithi043 ceph-mon[40255]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:21.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:21 smithi043 ceph-mon[40255]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-13T13:30:21.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:21 smithi043 ceph-mon[40255]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-13T13:30:23.837 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:23 smithi043 ceph-mon[40255]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:23.837 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:23 smithi043 ceph-mon[30106]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:23.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:23 smithi189 ceph-mon[32912]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:24.244 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:23 smithi043 systemd[1]: Stopping Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:30:24.568 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:24 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[74472]: ts=2024-09-13T13:30:24.269Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-13T13:30:24.568 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:24 smithi043 podman[75515]: 2024-09-13 13:30:24.290635326 +0000 UTC m=+0.142982108 container died 36cffb406b1fd54f2b1c9f4ae85c3f27322fce151509fc2882baf8d2b50af2e8 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:24.959 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:24 smithi043 podman[75515]: 2024-09-13 13:30:24.648878658 +0000 UTC m=+0.501225430 container cleanup 36cffb406b1fd54f2b1c9f4ae85c3f27322fce151509fc2882baf8d2b50af2e8 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:24.959 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:24 smithi043 bash[75515]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a 2024-09-13T13:30:25.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:24 smithi043 podman[75531]: 2024-09-13 13:30:24.959048903 +0000 UTC m=+0.661245737 container remove 36cffb406b1fd54f2b1c9f4ae85c3f27322fce151509fc2882baf8d2b50af2e8 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:25.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 podman[75531]: 2024-09-13 13:30:25.017913399 +0000 UTC m=+0.720110226 volume remove 86d305c50070b5089fc7d2a37ed5c29a0364bcdbac0a09b1c461ca463d6aab00 2024-09-13T13:30:25.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-13T13:30:25.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 systemd[1]: Stopped Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:30:25.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.033s CPU time. 2024-09-13T13:30:25.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 systemd[1]: Starting Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:30:25.643 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:30:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:30:25.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-mon[30106]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:25.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-mon[40255]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:25.713 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 podman[75621]: 2024-09-13 13:30:25.322132103 +0000 UTC m=+0.024263795 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-13T13:30:25.713 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 podman[75621]: 2024-09-13 13:30:25.436292872 +0000 UTC m=+0.138424554 volume create 2d201484b42aabd6a8dac3629ef783d7cd180dca4a67348867ef9ec5a32960f4 2024-09-13T13:30:25.713 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 podman[75621]: 2024-09-13 13:30:25.528325607 +0000 UTC m=+0.230457289 container create f08c22686d29776f3b0ba541876a82767d2a4b3f588f8559605fe919d78f5569 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:25.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:25 smithi189 ceph-mon[32912]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:30:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:30:25.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 podman[75621]: 2024-09-13 13:30:25.848123376 +0000 UTC m=+0.550255063 container init f08c22686d29776f3b0ba541876a82767d2a4b3f588f8559605fe919d78f5569 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:25.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 podman[75621]: 2024-09-13 13:30:25.853342518 +0000 UTC m=+0.555474201 container start f08c22686d29776f3b0ba541876a82767d2a4b3f588f8559605fe919d78f5569 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:25.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.869Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-13T13:30:25.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.869Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-13T13:30:25.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.870Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-09-13T13:30:25.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.870Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-13T13:30:25.993 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.904Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-13T13:30:25.993 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.905Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-13T13:30:25.993 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.908Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-13T13:30:25.993 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:25.908Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-13T13:30:25.993 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 bash[75621]: f08c22686d29776f3b0ba541876a82767d2a4b3f588f8559605fe919d78f5569 2024-09-13T13:30:25.993 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:25 smithi043 systemd[1]: Started Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:30:26.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:26.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:26 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:26.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:26 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:26 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:26 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:26.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:26.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:26 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:26.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:26 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:27.803 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:27 smithi189 ceph-mon[32912]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-13T13:30:27.803 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:27 smithi189 ceph-mon[32912]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:27.803 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:27 smithi189 ceph-mon[32912]: Reconfiguring daemon prometheus.a on smithi189 2024-09-13T13:30:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:27 smithi043 ceph-mon[30106]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-13T13:30:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:27 smithi043 ceph-mon[30106]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:27 smithi043 ceph-mon[30106]: Reconfiguring daemon prometheus.a on smithi189 2024-09-13T13:30:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:27 smithi043 ceph-mon[40255]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-13T13:30:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:27 smithi043 ceph-mon[40255]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:27 smithi043 ceph-mon[40255]: Reconfiguring daemon prometheus.a on smithi189 2024-09-13T13:30:27.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:27 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:27.871Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000868994s 2024-09-13T13:30:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:29 smithi189 ceph-mon[32912]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:29 smithi043 ceph-mon[30106]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:29 smithi043 ceph-mon[40255]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:31.935 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:31 smithi189 ceph-mon[32912]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:31 smithi043 ceph-mon[30106]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:31 smithi043 ceph-mon[40255]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:32.531 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 systemd[1]: Stopping Ceph prometheus.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:30:32.898 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-13T13:30:32.898 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-13T13:30:32.898 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-13T13:30:32.898 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-13T13:30:32.898 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-13T13:30:32.898 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-13T13:30:32.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-13T13:30:32.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-13T13:30:32.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.623Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-13T13:30:32.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.624Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-13T13:30:32.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.624Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-13T13:30:32.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[60871]: ts=2024-09-13T13:30:32.624Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-13T13:30:32.899 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 podman[63627]: 2024-09-13 13:30:32.633926634 +0000 UTC m=+0.141271065 container died a44705b2a1cd54e0b1b6631daebbf10a1f3f5c55cd16bbb956a71a7f70d2f846 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:33.197 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 podman[63627]: 2024-09-13 13:30:32.995930073 +0000 UTC m=+0.503274503 container cleanup a44705b2a1cd54e0b1b6631daebbf10a1f3f5c55cd16bbb956a71a7f70d2f846 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:33.197 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:32 smithi189 bash[63627]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a 2024-09-13T13:30:33.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 podman[63646]: 2024-09-13 13:30:33.197175513 +0000 UTC m=+0.565019349 container remove a44705b2a1cd54e0b1b6631daebbf10a1f3f5c55cd16bbb956a71a7f70d2f846 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:33.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-13T13:30:33.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 systemd[1]: Stopped Ceph prometheus.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:30:33.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.057s CPU time. 2024-09-13T13:30:33.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 systemd[1]: Starting Ceph prometheus.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:30:33.880 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-mon[32912]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:33.880 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 podman[63741]: 2024-09-13 13:30:33.532224545 +0000 UTC m=+0.017535697 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-13T13:30:33.881 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 podman[63741]: 2024-09-13 13:30:33.631313879 +0000 UTC m=+0.116625022 container create 72a37218f40208c469bfef73e1b586e9346d0bde13fc7b45c7d5ded6b7cd32e5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:33.881 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 podman[63741]: 2024-09-13 13:30:33.875461157 +0000 UTC m=+0.360772308 container init 72a37218f40208c469bfef73e1b586e9346d0bde13fc7b45c7d5ded6b7cd32e5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:33.909 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:33 smithi043 ceph-mon[30106]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:33.910 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:33 smithi043 ceph-mon[40255]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:34.148 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 podman[63741]: 2024-09-13 13:30:33.8801775 +0000 UTC m=+0.365488646 container start 72a37218f40208c469bfef73e1b586e9346d0bde13fc7b45c7d5ded6b7cd32e5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.899Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.900Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.900Z caller=main.go:623 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi189 (none))" 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.900Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.900Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.901Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.902Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.902Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.902Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-13T13:30:34.149 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.905Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.905Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.191µs 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.905Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.905Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.906Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.906Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=24.557µs wal_replay_duration=420.39µs wbl_replay_duration=134ns total_replay_duration=464.142µs 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.907Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.907Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.907Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.922Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.067393ms db_storage=757ns remote_storage=995ns web_handler=374ns query_engine=444ns scrape=399.617µs scrape_sd=94.766µs notify=10.059µs notify_sd=7.749µs rules=14.194198ms tracing=5.294µs 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.922Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:30:33.922Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-13T13:30:34.150 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 bash[63741]: 72a37218f40208c469bfef73e1b586e9346d0bde13fc7b45c7d5ded6b7cd32e5 2024-09-13T13:30:34.151 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:30:33 smithi189 systemd[1]: Started Ceph prometheus.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:30:34.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STOPPING 2024-09-13T13:30:34.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-13T13:30:34.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STOPPED 2024-09-13T13:30:34.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STARTING 2024-09-13T13:30:34.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Serving on http://:::9283 2024-09-13T13:30:34.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STARTED 2024-09-13T13:30:34.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STOPPING 2024-09-13T13:30:34.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-13T13:30:34.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STOPPED 2024-09-13T13:30:34.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STARTING 2024-09-13T13:30:34.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Serving on http://:::9283 2024-09-13T13:30:34.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STARTED 2024-09-13T13:30:34.493 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STOPPING 2024-09-13T13:30:35.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-13T13:30:35.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-13T13:30:35.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"} : dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"} : dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"} : dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]} : dispatch 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]} : dispatch 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STOPPED 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STARTING 2024-09-13T13:30:35.223 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Serving on http://:::9283 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:34 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:30:34] ENGINE Bus STARTED 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"} : dispatch 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.224 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"} : dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"} : dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:35.225 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]} : dispatch 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]} : dispatch 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.226 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:30:35.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"} : dispatch 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"} : dispatch 2024-09-13T13:30:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"} : dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]} : dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]} : dispatch 2024-09-13T13:30:35.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-13T13:30:35.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:35.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:35.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:30:35.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-13T13:30:35.876 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:30:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:30:35.874Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003441791s 2024-09-13T13:30:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:36.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[30106]: osdmap e64: 8 total, 8 up, 8 in 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-13T13:30:36.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[40255]: osdmap e64: 8 total, 8 up, 8 in 2024-09-13T13:30:36.475 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:36.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:36 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-13T13:30:36.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:36 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-13T13:30:36.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:36 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-13T13:30:36.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:36 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-13T13:30:36.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:36 smithi189 ceph-mon[32912]: osdmap e64: 8 total, 8 up, 8 in 2024-09-13T13:30:36.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:37.096 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:37 smithi043 ceph-mon[30106]: osdmap e65: 8 total, 8 up, 8 in 2024-09-13T13:30:37.097 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:37 smithi043 ceph-mon[30106]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:37.393 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:37 smithi189 ceph-mon[32912]: osdmap e65: 8 total, 8 up, 8 in 2024-09-13T13:30:37.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:37 smithi189 ceph-mon[32912]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:37 smithi043 ceph-mon[40255]: osdmap e65: 8 total, 8 up, 8 in 2024-09-13T13:30:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:37 smithi043 ceph-mon[40255]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:38 smithi189 ceph-mon[32912]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-13T13:30:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:38 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:38.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:38 smithi043 ceph-mon[30106]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-13T13:30:38.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:38.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:38 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:38.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:38 smithi043 ceph-mon[40255]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-13T13:30:38.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:38.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:38 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:39 smithi189 ceph-mon[32912]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:30:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:39 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:39 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:39.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:39 smithi043 ceph-mon[40255]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:30:39.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:39 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:39.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:39 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:39 smithi043 ceph-mon[30106]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:30:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:39 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:39 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:40 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:30:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:40 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:30:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:40 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:40 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:30:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:40 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:30:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:40 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:40 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:30:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:40 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:30:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:40 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:30:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:41 smithi189 ceph-mon[32912]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-13T13:30:41.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:41 smithi043 ceph-mon[30106]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-13T13:30:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:41 smithi043 ceph-mon[40255]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-13T13:30:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:42 smithi189 ceph-mon[32912]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-13T13:30:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:42 smithi189 ceph-mon[32912]: Cluster is now healthy 2024-09-13T13:30:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:42 smithi043 ceph-mon[30106]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-13T13:30:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:42 smithi043 ceph-mon[30106]: Cluster is now healthy 2024-09-13T13:30:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:42 smithi043 ceph-mon[40255]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-13T13:30:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:42 smithi043 ceph-mon[40255]: Cluster is now healthy 2024-09-13T13:30:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:43 smithi189 ceph-mon[32912]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-13T13:30:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:43 smithi043 ceph-mon[30106]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-13T13:30:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:43 smithi043 ceph-mon[40255]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-13T13:30:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:45 smithi043 ceph-mon[30106]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-09-13T13:30:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:45 smithi043 ceph-mon[40255]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-09-13T13:30:45.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:45 smithi189 ceph-mon[32912]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-09-13T13:30:45.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:30:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:30:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:30:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:30:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:47 smithi189 ceph-mon[32912]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s; 32 B/s, 1 objects/s recovering 2024-09-13T13:30:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:47 smithi043 ceph-mon[30106]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s; 32 B/s, 1 objects/s recovering 2024-09-13T13:30:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:47 smithi043 ceph-mon[40255]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s; 32 B/s, 1 objects/s recovering 2024-09-13T13:30:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:49 smithi189 ceph-mon[32912]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-09-13T13:30:49.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:49 smithi043 ceph-mon[30106]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-09-13T13:30:49.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:49 smithi043 ceph-mon[40255]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-09-13T13:30:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:50.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:50.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:30:51.549 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:51 smithi189 ceph-mon[32912]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-13T13:30:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:51 smithi043 ceph-mon[30106]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-13T13:30:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:51 smithi043 ceph-mon[40255]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-13T13:30:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:53 smithi189 ceph-mon[32912]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:53 smithi043 ceph-mon[30106]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:53 smithi043 ceph-mon[40255]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:54.592 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 91% (11621/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to 'b0201b8c79733293453e7f10a10c7fa43119222b'. 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-13T13:30:55.381 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-13T13:30:55.382 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at b0201b8c797 Merge branch 'wip-aclamk-cbt-combined' of https://github.com/aclamk/ceph into wip-guits-main-2024-09-13-0840 2024-09-13T13:30:55.387 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-13T13:30:55.412 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T13:30:55.414 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T13:30:55.414 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T13:30:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:55 smithi189 ceph-mon[32912]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:55.649 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:30:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:30:55.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:55 smithi043 ceph-mon[30106]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:55.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:55 smithi043 ceph-mon[40255]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:30:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:30:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:30:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:30:56.154 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T13:30:56.230 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T13:30:56.271 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T13:30:56.273 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T13:30:56.273 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T13:30:56.313 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T13:30:56.316 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T13:30:56.316 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-13T13:30:56.386 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-13T13:30:56.388 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-13T13:30:56.388 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b0201b8c79733293453e7f10a10c7fa43119222b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ parallel=1 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' '' = --serial ']' 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ crimson=0 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' '' = --crimson ']' 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ color= 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -t 1 ']' 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-13T13:30:56.518 INFO:tasks.workunit.client.0.smithi043.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-13T13:30:56.519 INFO:tasks.workunit.client.0.smithi043.stderr:+ declare -A pids 2024-09-13T13:30:56.519 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.519 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.520 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio 2024-09-13T13:30:56.520 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio' 2024-09-13T13:30:56.520 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio 2024-09-13T13:30:56.520 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.521 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio 2024-09-13T13:30:56.521 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76243 2024-09-13T13:30:56.521 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio on pid 76243' 2024-09-13T13:30:56.521 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio on pid 76243 2024-09-13T13:30:56.523 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76243 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio_pp 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio_pp' 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio_pp 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.524 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.525 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio_pp 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76250 2024-09-13T13:30:56.526 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio_pp on pid 76250 2024-09-13T13:30:56.529 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio_pp on pid 76250' 2024-09-13T13:30:56.529 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76250 2024-09-13T13:30:56.529 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.529 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-13T13:30:56.529 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.530 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.531 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.532 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76259 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io on pid 76259' 2024-09-13T13:30:56.533 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io on pid 76259 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76259 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io_pp 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io_pp' 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io_pp 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.537 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.538 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.540 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io_pp 2024-09-13T13:30:56.541 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76270 2024-09-13T13:30:56.542 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io_pp on pid 76270 2024-09-13T13:30:56.546 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io_pp on pid 76270' 2024-09-13T13:30:56.546 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76270 2024-09-13T13:30:56.546 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.546 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.546 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-13T13:30:56.546 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_asio 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_asio' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_asio 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.547 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.548 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.549 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.550 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.551 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_asio 2024-09-13T13:30:56.552 INFO:tasks.workunit.client.0.smithi043.stdout:test api_asio on pid 76284 2024-09-13T13:30:56.558 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76284 2024-09-13T13:30:56.558 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_asio on pid 76284' 2024-09-13T13:30:56.558 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76284 2024-09-13T13:30:56.558 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.558 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.558 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_list 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_list' 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_list 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.559 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.560 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.561 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.562 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.563 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.564 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.565 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76243/exe 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_list 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76301 2024-09-13T13:30:56.566 INFO:tasks.workunit.client.0.smithi043.stdout:test api_list on pid 76301 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_list on pid 76301' 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76301 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.573 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.574 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.576 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.578 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.579 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.580 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.581 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio: /' 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76250/exe 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock 2024-09-13T13:30:56.582 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock on pid 76322 2024-09-13T13:30:56.585 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76322 2024-09-13T13:30:56.585 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock on pid 76322' 2024-09-13T13:30:56.585 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76322 2024-09-13T13:30:56.585 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock_pp 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock_pp' 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.586 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock_pp 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.587 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.588 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.589 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.590 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-13T13:30:56.591 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.595 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock_pp on pid 76340 2024-09-13T13:30:56.599 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock_pp 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76340 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock_pp on pid 76340' 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76340 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc' 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76259/exe 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.600 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.601 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io: /' 2024-09-13T13:30:56.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76360 2024-09-13T13:30:56.608 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc on pid 76360 2024-09-13T13:30:56.613 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc on pid 76360' 2024-09-13T13:30:56.613 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76360 2024-09-13T13:30:56.613 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.613 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.613 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.613 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-13T13:30:56.613 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76270/exe 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.614 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc_pp 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc_pp' 2024-09-13T13:30:56.616 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io.log 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-09-13T13:30:56.618 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.622 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.623 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.624 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76284/exe 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.625 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc_pp 2024-09-13T13:30:56.626 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc_pp on pid 76385 2024-09-13T13:30:56.631 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc_pp 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76385 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc_pp on pid 76385' 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76385 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_tier_pp 2024-09-13T13:30:56.632 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_tier_pp' 2024-09-13T13:30:56.634 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.635 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_asio: /' 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.636 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-13T13:30:56.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.645 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76322/exe 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.646 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.647 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_tier_pp 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.648 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76340/exe 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.650 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.651 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76301/exe 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-09-13T13:30:56.653 INFO:tasks.workunit.client.0.smithi043.stdout:test api_tier_pp on pid 76435 2024-09-13T13:30:56.655 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:56.657 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_tier_pp 2024-09-13T13:30:56.657 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76435 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_tier_pp on pid 76435' 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76435 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock: /' 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_list: /' 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_pool 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_pool' 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_list.log 2024-09-13T13:30:56.658 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76360/exe 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.660 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc: /' 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.661 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_pool 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_pool 2024-09-13T13:30:56.663 INFO:tasks.workunit.client.0.smithi043.stdout:test api_pool on pid 76498 2024-09-13T13:30:56.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76498 2024-09-13T13:30:56.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_pool on pid 76498' 2024-09-13T13:30:56.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76498 2024-09-13T13:30:56.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.666 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.667 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots' 2024-09-13T13:30:56.668 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.669 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.670 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.671 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76385/exe 2024-09-13T13:30:56.672 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots on pid 76534 2024-09-13T13:30:56.676 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots 2024-09-13T13:30:56.676 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76534 2024-09-13T13:30:56.676 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots on pid 76534' 2024-09-13T13:30:56.676 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76534 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots_pp 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots_pp' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots_pp 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.685 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.686 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots_pp on pid 76602 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots_pp 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76602 2024-09-13T13:30:56.688 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots_pp on pid 76602' 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76602 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat 2024-09-13T13:30:56.690 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.691 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat 2024-09-13T13:30:56.692 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.693 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat on pid 76616 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76616 2024-09-13T13:30:56.696 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat on pid 76616' 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76616 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76498/exe 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76435/exe 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_pool: /' 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat_pp 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat_pp' 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.702 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.703 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76534/exe 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat_pp 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.707 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.708 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat_pp 2024-09-13T13:30:56.709 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat_pp on pid 76654 2024-09-13T13:30:56.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76654 2024-09-13T13:30:56.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat_pp on pid 76654' 2024-09-13T13:30:56.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76654 2024-09-13T13:30:56.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.717 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify on pid 76680 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76680 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify on pid 76680' 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76680 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify_pp 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify_pp' 2024-09-13T13:30:56.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify_pp 2024-09-13T13:30:56.723 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76602/exe 2024-09-13T13:30:56.724 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify_pp on pid 76695 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify_pp 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76695 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify_pp on pid 76695' 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76695 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd' 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-13T13:30:56.728 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.736 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.737 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.738 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd on pid 76726 2024-09-13T13:30:56.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76726 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.740 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd on pid 76726' 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76726 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd_pp 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd_pp' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd_pp 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd_pp 2024-09-13T13:30:56.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.745 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76740 2024-09-13T13:30:56.745 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd_pp on pid 76740 2024-09-13T13:30:56.746 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd_pp on pid 76740' 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76740 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service' 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.748 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.749 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.750 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.751 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service 2024-09-13T13:30:56.753 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.754 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service on pid 76759 2024-09-13T13:30:56.759 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76759 2024-09-13T13:30:56.759 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service on pid 76759' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76759 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76616/exe 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service_pp 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service_pp' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat: /' 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.766 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76654/exe 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.771 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76680/exe 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service_pp 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.779 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service_pp 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76864 2024-09-13T13:30:56.782 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service_pp on pid 76864 2024-09-13T13:30:56.786 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service_pp on pid 76864' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76864 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.787 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_write_operations 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_write_operations' 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_write_operations 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76695/exe 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76726/exe 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.797 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-13T13:30:56.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76740/exe 2024-09-13T13:30:56.800 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_write_operations on pid 76894 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_write_operations 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76894 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_write_operations on pid 76894' 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76894 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_read_operations 2024-09-13T13:30:56.803 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_read_operations' 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd: /' 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:56.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:30:56.809 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-13T13:30:56.809 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.809 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76759/exe 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.813 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service: /' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.817 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service.log 2024-09-13T13:30:56.817 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.817 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_read_operations 2024-09-13T13:30:56.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.817 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_read_operations on pid 76932 2024-09-13T13:30:56.822 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_read_operations 2024-09-13T13:30:56.822 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76932 2024-09-13T13:30:56.822 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_read_operations on pid 76932' 2024-09-13T13:30:56.822 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76932 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s list_parallel 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' list_parallel' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.826 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo list_parallel 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.830 INFO:tasks.workunit.client.0.smithi043.stdout:test list_parallel on pid 76960 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=list_parallel 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76960 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test list_parallel on pid 76960' 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76960 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76864/exe 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s open_pools_parallel 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' open_pools_parallel' 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo open_pools_parallel 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.832 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-13T13:30:56.833 INFO:tasks.workunit.client.0.smithi043.stdout:test open_pools_parallel on pid 76985 2024-09-13T13:30:56.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=open_pools_parallel 2024-09-13T13:30:56.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76985 2024-09-13T13:30:56.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test open_pools_parallel on pid 76985' 2024-09-13T13:30:56.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=76985 2024-09-13T13:30:56.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s delete_pools_parallel 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' delete_pools_parallel' 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.845 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76894/exe 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo delete_pools_parallel 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.850 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-13T13:30:56.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76932/exe 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.855 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.856 INFO:tasks.workunit.client.0.smithi043.stdout:test delete_pools_parallel on pid 77074 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=delete_pools_parallel 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.858 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77074 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test delete_pools_parallel on pid 77074' 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77074 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.859 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s cls 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' cls' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo cls 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.866 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=cls 2024-09-13T13:30:56.866 INFO:tasks.workunit.client.0.smithi043.stdout:test cls on pid 77102 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77102 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test cls on pid 77102' 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77102 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76985/exe 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s cmd 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' cmd' 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.870 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.871 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo cmd 2024-09-13T13:30:56.872 INFO:tasks.workunit.client.0.smithi043.stdout:test cmd on pid 77130 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=cmd 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77130 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test cmd on pid 77130' 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77130 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s handler_error 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' handler_error' 2024-09-13T13:30:56.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo handler_error 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.877 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.877 INFO:tasks.workunit.client.0.smithi043.stdout:test handler_error on pid 77142 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=handler_error 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.878 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77142 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test handler_error on pid 77142' 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77142 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.879 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.880 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.880 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.880 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s io 2024-09-13T13:30:56.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' io' 2024-09-13T13:30:56.880 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.880 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo io 2024-09-13T13:30:56.880 INFO:tasks.workunit.client.0.smithi043.stdout:test io on pid 77152 2024-09-13T13:30:56.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=io 2024-09-13T13:30:56.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77152 2024-09-13T13:30:56.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test io on pid 77152' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77152 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s ec_io 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' ec_io' 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo ec_io 2024-09-13T13:30:56.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-09-13T13:30:56.884 INFO:tasks.workunit.client.0.smithi043.stdout:test ec_io on pid 77160 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=ec_io 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77160 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test ec_io on pid 77160' 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77160 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.896 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.897 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s list 2024-09-13T13:30:56.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' list' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/76960/exe 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.903 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.904 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77074/exe 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.905 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ list_parallel: /' 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo list 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=list 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77257 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test list on pid 77257' 2024-09-13T13:30:56.914 INFO:tasks.workunit.client.0.smithi043.stdout:test list on pid 77257 2024-09-13T13:30:56.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77257 2024-09-13T13:30:56.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-09-13T13:30:56.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s ec_list 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' ec_list' 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo ec_list 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.927 INFO:tasks.workunit.client.0.smithi043.stdout:test ec_list on pid 77313 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=ec_list 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77313 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test ec_list on pid 77313' 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77313 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s misc 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' misc' 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo misc 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77142/exe 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.935 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77102/exe 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_handler_error.log 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ handler_error: /' 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.938 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_cls 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_cls.log 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.940 INFO:tasks.workunit.client.0.smithi043.stdout:test misc on pid 77361 2024-09-13T13:30:56.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=misc 2024-09-13T13:30:56.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77361 2024-09-13T13:30:56.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test misc on pid 77361' 2024-09-13T13:30:56.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77361 2024-09-13T13:30:56.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ cls: /' 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_handler_error 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77160/exe 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s pool 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77130/exe 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' pool' 2024-09-13T13:30:56.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77152/exe 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ ec_io: /' 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.964 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_ec_io.log 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_ec_io 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.965 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ cmd: /' 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.966 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.967 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77313/exe 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77257/exe 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.972 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_ec_list.log 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ ec_list: /' 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_io 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:56.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo pool 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_list.log 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_cmd 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ list: /' 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_list 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=pool 2024-09-13T13:30:56.974 INFO:tasks.workunit.client.0.smithi043.stdout:test pool on pid 77453 2024-09-13T13:30:56.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77453 2024-09-13T13:30:56.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test pool on pid 77453' 2024-09-13T13:30:56.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77453 2024-09-13T13:30:56.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_cmd.log 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ io: /' 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_ec_list 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_io.log 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s read_operations 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' read_operations' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo read_operations 2024-09-13T13:30:56.981 INFO:tasks.workunit.client.0.smithi043.stdout:test read_operations on pid 77495 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=read_operations 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77495 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test read_operations on pid 77495' 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77495 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s snapshots 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' snapshots' 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo snapshots 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.985 INFO:tasks.workunit.client.0.smithi043.stdout:test snapshots on pid 77529 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=snapshots 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77529 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test snapshots on pid 77529' 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77529 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s watch_notify 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' watch_notify' 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:56.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo watch_notify 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:56.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:56.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:56.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:56.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:56.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:56.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:56.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:56.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:56.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:56.999 INFO:tasks.workunit.client.0.smithi043.stdout:test watch_notify on pid 77606 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=watch_notify 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77606 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test watch_notify on pid 77606' 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77606 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:57.004 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s write_operations 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:57.005 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' write_operations' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo write_operations 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:57.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77361/exe 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77453/exe 2024-09-13T13:30:57.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ misc: /' 2024-09-13T13:30:57.014 INFO:tasks.workunit.client.0.smithi043.stdout:test write_operations on pid 77659 2024-09-13T13:30:57.018 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=write_operations 2024-09-13T13:30:57.018 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77659 2024-09-13T13:30:57.018 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test write_operations on pid 77659' 2024-09-13T13:30:57.018 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=77659 2024-09-13T13:30:57.018 INFO:tasks.workunit.client.0.smithi043.stderr:+ ret=0 2024-09-13T13:30:57.018 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76435 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76435 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:57.019 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_misc 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:57.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_misc.log 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:57.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:57.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_pool 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_pool.log 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ pool: /' 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-13T13:30:57.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:57.026 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:57.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77529/exe 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:57.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-13T13:30:57.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-13T13:30:57.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:57.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77495/exe 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:57.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_snapshots 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_snapshots.log 2024-09-13T13:30:57.033 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ snapshots: /' 2024-09-13T13:30:57.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:57.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:57.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:57.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:57.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-13T13:30:57.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_read_operations.log 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_read_operations 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ read_operations: /' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-13T13:30:57.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-13T13:30:57.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:57.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77606/exe 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-09-13T13:30:57.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/77659/exe 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:57.047 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_watch_notify 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ watch_notify: /' 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-09-13T13:30:57.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_write_operations 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_write_operations.log 2024-09-13T13:30:57.049 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ write_operations: /' 2024-09-13T13:30:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2168266667' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-13T13:30:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2168266667' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-13T13:30:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:30:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2168266667' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:57.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:30:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: osdmap e66: 8 total, 8 up, 8 in 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/677741889' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-76354-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4064756052' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2761042371' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3573117975' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2538081137' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/817171001' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4040901691' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/448620130' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/608815737' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4031911592' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1240452481' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-76627-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1072210978' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/881511846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-76776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4287923658' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-76803-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4009850590' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:58.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1979600777' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3180977912' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-13T13:30:58.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment set-up. 2024-09-13T13:30:58.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-13T13:30:58.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-13T13:30:58.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-09-13T13:30:58.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-13T13:30:58.701 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (24 ms) 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (4 ms) 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (22 ms) 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (0 ms) 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-13T13:30:58.702 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (2 ms) 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (3 ms) 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (3 ms) 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (2 ms) 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados (63 ms total) 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment tear-down 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (2077 ms total) 2024-09-13T13:30:58.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: osdmap e66: 8 total, 8 up, 8 in 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/677741889' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-76354-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4064756052' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2761042371' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3573117975' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2538081137' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/817171001' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4040901691' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/448620130' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/608815737' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4031911592' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1240452481' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-76627-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1072210978' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/881511846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-76776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4287923658' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-76803-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4009850590' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-77444-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-77385-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: osdmap e66: 8 total, 8 up, 8 in 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/677741889' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-76354-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4064756052' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2761042371' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3573117975' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2538081137' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/817171001' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4040901691' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/448620130' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/608815737' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4031911592' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1240452481' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-76627-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1072210978' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/881511846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-76776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4287923658' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-76803-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4009850590' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1979600777' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3180977912' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1979600777' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3180977912' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:58.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:59.017 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-13T13:30:59.017 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment set-up. 2024-09-13T13:30:59.017 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-13T13:30:59.017 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-13T13:30:59.017 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjects (428 ms) 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (39 ms) 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-09-13T13:30:59.018 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (85 ms) 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-09-13T13:30:59.019 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-09-13T13:30:59.020 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (72 ms) 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: x cursor=MIN 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=1 cursor=14:02547ec2:::1:head 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=10 cursor=14:52ea6a34:::10:head 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=13 cursor=14:566253c9:::13:head 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=7 cursor=14:5c6b0b28:::7:head 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=14 cursor=14:62a1935d:::14:head 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=0 cursor=14:6cac518f:::0:head 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=15 cursor=14:863748b0:::15:head 2024-09-13T13:30:59.021 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=11 cursor=14:89d3ae78:::11:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=5 cursor=14:b29083e3:::5:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=8 cursor=14:bd63b0f1:::8:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=6 cursor=14:c4fdafeb:::6:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=3 cursor=14:cfc208b3:::3:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=4 cursor=14:d83876eb:::4:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=12 cursor=14:de5d7c5f:::12:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=9 cursor=14:e960b815:::9:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=2 cursor=14:f905c69b:::2:head 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-13T13:30:59.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=1 cursor=14:02547ec2:::1:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=10 cursor=14:52ea6a34:::10:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=13 cursor=14:566253c9:::13:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:566253c9:::13:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=7 cursor=14:5c6b0b28:::7:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-09-13T13:30:59.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-09-13T13:30:59.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=14 cursor=14:62a1935d:::14:head 2024-09-13T13:30:59.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-09-13T13:30:59.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-09-13T13:30:59.078 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=0 cursor=14:6cac518f:::0:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=15 cursor=14:863748b0:::15:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:863748b0:::15:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=11 cursor=14:89d3ae78:::11:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=5 cursor=14:b29083e3:::5:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=8 cursor=14:bd63b0f1:::8:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-09-13T13:30:59.079 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=6 cursor=14:c4fdafeb:::6:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=3 cursor=14:cfc208b3:::3:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=4 cursor=14:d83876eb:::4:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=12 cursor=14:de5d7c5f:::12:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-09-13T13:30:59.080 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=9 cursor=14:e960b815:::9:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:e960b815:::9:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=2 cursor=14:f905c69b:::2:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:e960b815:::9:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:e960b815:::9:head expected=14:e960b815:::9:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=9 expected=9 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:5c6b0b28:::7:head expected=14:5c6b0b28:::7:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=7 expected=7 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-09-13T13:30:59.081 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:f905c69b:::2:head expected=14:f905c69b:::2:head 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=2 expected=2 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:bd63b0f1:::8:head expected=14:bd63b0f1:::8:head 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=8 expected=8 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:b29083e3:::5:head expected=14:b29083e3:::5:head 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=5 expected=5 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:de5d7c5f:::12:head expected=14:de5d7c5f:::12:head 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-09-13T13:30:59.082 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=12 expected=12 2024-09-13T13:30:59.083 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-09-13T13:30:59.083 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:d83876eb:::4:head expected=14:d83876eb:::4:head 2024-09-13T13:30:59.083 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-09-13T13:30:59.083 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=4 expected=4 2024-09-13T13:30:59.083 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-09-13T13:30:59.083 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:cfc208b3:::3:head expected=14:cfc208b3:::3:head 2024-09-13T13:30:59.083 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-09-13T13:30:59.275 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=3 expecte list_parallel: process_1_[77328]: starting. 2024-09-13T13:30:59.275 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[77328]: creating pool ceph_test_rados_list_parallel.smithi043-77210 2024-09-13T13:30:59.275 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[77328]: created object 0... 2024-09-13T13:30:59.275 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[77328]: created object 25... 2024-09-13T13:30:59.275 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[77328]: created object 49... 2024-09-13T13:30:59.275 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[77328]: finishing. 2024-09-13T13:30:59.275 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[77328]: shutting down. 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[77330]: starting. 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[77330]: listing objects. 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[77330]: listed object 0... 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[77330]: listed object 25... 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[77330]: saw 50 objects 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[77330]: shutting down. 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[77915]: starting. 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[77915]: creating pool ceph_test_rados_list_parallel.smithi043-77210 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[77915]: created object 0... 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[77915]: created object 25... 2024-09-13T13:30:59.276 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[77915]: created object 49... 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[77915]: finishing. 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[77915]: shutting down. 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[77916]: starting. 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[77916]: listing objects. 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[77916]: listed object 0... 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[77916]: listed object 25... 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[77916]: saw 42 objects 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[77916]: shutting down. 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[77917]: starting. 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[77917]: removed 25 objects... 2024-09-13T13:30:59.277 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[77917]: removed half of the objects 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[77917]: removed 50 objects... 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[77917]: removed 50 objects 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[77917]: shutting down. 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[77963]: starting. 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[77963]: creating pool ceph_test_rados_list_parallel.smithi043-77210 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[77963]: created object 0... 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[77963]: created object 25... 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[77963]: created object 49... 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[77963]: finishing. 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[77963]: shutting down. 2024-09-13T13:30:59.278 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[77964]: starting. 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[77964]: listing objects. 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[77964]: listed object 0... 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[77964]: listed object 25... 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[77964]: listed object 50... 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[77964]: saw 56 objects 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[77964]: shutting down. 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[77965]: starting. 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[77965]: added 25 objects... 2024-09-13T13:30:59.279 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[77965]: added half of the objects 2024-09-13T13:30:59.280 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[77965]: added 50 objects... 2024-09-13T13:30:59.280 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[77965]: added 50 objects 2024-09-13T13:30:59.280 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[77965]: shutting down. 2024-09-13T13:30:59.280 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.280 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ********************** handler_error: Running main() from gmock_main.cc 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] Global test environment set-up. 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (2368 ms) 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] 1 test from neocls_handler_error (2368 ms total) 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] Global test environment tear-down 2024-09-13T13:30:59.303 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [==========] 1 test from 1 test suite ran. (2368 ms total) 2024-09-13T13:30:59.304 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ PASSED ] 1 test. 2024-09-13T13:30:59.308 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (61 ms) 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (56 ms) 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2423 ms) 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (2540 ms total) 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: 2024-09-13T13:30:59.309 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-13T13:30:59.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (2540 ms total) 2024-09-13T13:30:59.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: Running main() from gmock_main.cc 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [==========] Running 1 test from 1 test suite. 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] Global test environment set-up. 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] 1 test from NeoRadosCls 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ OK ] NeoRadosCls.DNE (2398 ms) 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] 1 test from NeoRadosCls (2398 ms total) 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] Global test environment tear-down 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [==========] 1 test from 1 test suite ran. (2398 ms total) 2024-09-13T13:30:59.333 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ PASSED ] 1 test. 2024-09-13T13:30:59.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-09-13T13:30:59.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-09-13T13:30:59.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-13T13:30:59.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-09-13T13:30:59.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (364 ms) 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (17 ms) 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (13 ms) 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (32 ms) 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (8 ms) 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-13T13:30:59.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (9 ms) 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (8 ms) 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (6 ms) 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-09-13T13:30:59.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (13 ms) 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (8 ms) 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (3 ms) 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (514 ms total) 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (2497 ms total) 2024-09-13T13:30:59.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (68 ms) 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (54 ms) 2024-09-13T13:30:59.489 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2441 ms) 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.283227+0000 mon.a [WRN] Health check failed: 34 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286136+0000 mon.a [INF] from='client.? 172.21.15.43:0/677741889' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-76354-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286252+0000 mon.a [INF] from='client.? 172.21.15.43:0/1240452481' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-76627-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286343+0000 mon.a [INF] from='client.? 172.21.15.43:0/1072210978' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286439+0000 mon.a [INF] from='client.? 172.21.15.43:0/881511846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-76776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286537+0000 mon.a [INF] from='client.? 172.21.15.43:0/4287923658' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-76803-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286629+0000 mon.a [INF] from='client.? 172.21.15.43:0/4009850590' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286704+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286807+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.490 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.286908+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.491 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287012+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.491 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287085+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: pgmap v48: 1220 pgs: 1088 unknown, 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:30:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: Health check failed: 34 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:30:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/677741889' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-76354-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1240452481' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-76627-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1072210978' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/881511846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-76776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4287923658' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-76803-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4009850590' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: osdmap e67: 8 total, 8 up, 8 in 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: osdmap e68: 8 total, 8 up, 8 in 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:59.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:30:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: pgmap v48: 1220 pgs: 1088 unknown, 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: Health check failed: 34 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/677741889' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-76354-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1240452481' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-76627-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1072210978' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/881511846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-76776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4287923658' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-76803-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4009850590' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: pgmap v48: 1220 pgs: 1088 unknown, 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: Health check failed: 34 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/677741889' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-76354-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1240452481' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-76627-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1072210978' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/881511846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-76776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4287923658' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-76803-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4009850590' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-76432-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-77079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-76650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-76709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: osdmap e67: 8 total, 8 up, 8 in 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: osdmap e68: 8 total, 8 up, 8 in 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: osdmap e67: 8 total, 8 up, 8 in 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: osdmap e68: 8 total, 8 up, 8 in 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:59.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:30:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287161+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rado********* 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[78011]: starting. 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[78011]: creating pool ceph_test_rados_list_parallel.smithi043-77210 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[78011]: created object 0... 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[78011]: created object 25... 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[78011]: created object 49... 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[78011]: finishing. 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[78011]: shutting down. 2024-09-13T13:30:59.753 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: starting. 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: listing objects. 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: listed object 0... 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: listed object 25... 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: listed object 50... 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: listed object 75... 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: saw 97 objects 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[78012]: shutting down. 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.754 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[78015]: starting. 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[78015]: removed 25 objects... 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[78015]: removed half of the objects 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[78015]: removed 50 objects... 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[78015]: removed 50 objects 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[78015]: shutting down. 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[78013]: starting. 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[78013]: added 25 objects... 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[78013]: added half of the objects 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[78013]: added 50 objects... 2024-09-13T13:30:59.755 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[78013]: added 50 objects 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[78013]: shutting down. 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[78014]: starting. 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[78014]: added 25 objects... 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[78014]: added half of the objects 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[78014]: added 50 objects... 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[78014]: added 50 objects 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[78014]: shutting down. 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.756 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[78393]: starting. 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[78393]: creating pool ceph_test_rados_list_parallel.smithi043-77210 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[78393]: created object 0... 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[78393]: created object 25... 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[78393]: created object 49... 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[78393]: finishing. 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[78393]: shutting down. 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:30:59.757 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: starting. 2024-09-13T13:30:59.758 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: listing objects. 2024-09-13T13:30:59.758 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: listed object 0... 2024-09-13T13:31:00.322 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: listed object 25... 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: listed object 50... 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: listed object 75... 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: listed object 100... 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: listed object 125... 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: saw 150 objects 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[78394]: shutting down. 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[78395]: starting. 2024-09-13T13:31:00.323 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[78395]: added 25 objects... 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[78395]: added half of the objects 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[78395]: added 50 objects... 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[78395]: added 50 objects 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[78395]: shutting down. 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-09-13T13:31:00.324 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******* SUCCESS ********** 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout:s-api-smithi043-76896-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287255+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-76332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287319+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-76371-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287404+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-76844-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287471+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-76315-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287527+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-76423-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287584+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-76417-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:58.287640+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.291303+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-76888-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.291428+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-77385-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:00.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.291523+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-77444-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.299653+0000 mon.b [INF] from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.313249+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.410504+0000 mon.b [INF] from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.413093+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.426249+0000 mon.b [INF] from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.434882+0000 mon.b [INF] from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.450508+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.454507+0000 mon.b [INF] from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.467962+0000 mon.b [INF] from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.473110+0000 mon.b [INF] from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.473897+0000 client.admin [INF] onexx 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.488354+0000 mon.c [INF] from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.507621+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.507899+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.508247+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.511499+0000 mon.b [INF] from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.513644+0000 mon.c [INF] from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.525889+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.527536+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.527824+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.530631+0000 mon.b [INF] from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.531389+0000 mon.c [INF] from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.536330+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.536709+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.539872+0000 mon.b [INF] from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.549863+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.550237+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.553862+0000 mon.b [INF] from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:30:59.554289+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.295382+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.295504+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.295573+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.295620+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.295684+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.300829+0000 mon.b [INF] from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.301010+0000 mon.b [INF] from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.301157+0000 mon.b [INF] from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.307306+0000 mon.c [INF] from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.310989+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.312971+0000 mon.c [INF] from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.314611+0000 mon.c [INF] from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.314811+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.315449+0000 mon.a [INF] from='client.? 172.21.15.43:0/2575316564' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.315719+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.319827+0000 mon.b [INF] from='client.? 172.21.15.43:0/3894889994' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.320206+0000 mon.b [INF] from='client.? 172.21.15.43:0/2514411' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.323178+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.323415+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.323760+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.324249+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: onexx 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: osdmap e69: 8 total, 8 up, 8 in 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2575316564' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3894889994' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2514411' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: onexx 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: onexx 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: osdmap e69: 8 total, 8 up, 8 in 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2575316564' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3894889994' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2514411' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:00.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-76776-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-76354-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: osdmap e69: 8 total, 8 up, 8 in 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2575316564' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3894889994' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2514411' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:00.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:01.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: pgmap v51: 636 pgs: 152 creating+peering, 484 active+clean; 609 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 90 KiB/s wr, 374 op/s 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: twoxx 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2575316564' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-13T13:31:01.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: osdmap e70: 8 total, 8 up, 8 in 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:01.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:01.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:01.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:01.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: pgmap v51: 636 pgs: 152 creating+peering, 484 active+clean; 609 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 90 KiB/s wr, 374 op/s 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: twoxx 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2575316564' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: osdmap e70: 8 total, 8 up, 8 in 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: pgmap v51: 636 pgs: 152 creating+peering, 484 active+clean; 609 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 90 KiB/s wr, 374 op/s 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-13T13:31:01.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: twoxx 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2575316564' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-76315-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/247897181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-13T13:31:01.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: osdmap e70: 8 total, 8 up, 8 in 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:01.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:01.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:00.324505+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-2","app": "rados","yes_i_really_me open_pools_parallel: process_1_[77166]: starting. 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[77166]: creating pool ceph_test_rados_open_pools_parallel.smithi043-77112 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[77166]: created object 0... 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[77166]: created object 25... 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[77166]: created object 49... 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[77166]: finishing. 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[77166]: shutting down. 2024-09-13T13:31:02.326 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[77170]: starting. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[77170]: rados_pool_create. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[77170]: rados_ioctx_create. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[77170]: shutting down. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[78091]: starting. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[78091]: creating pool ceph_test_rados_open_pools_parallel.smithi043-77112 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[78091]: created object 0... 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[78091]: created object 25... 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[78091]: created object 49... 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[78091]: finishing. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[78091]: shutting down. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[78092]: starting. 2024-09-13T13:31:02.327 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[78092]: rados_pool_create. 2024-09-13T13:31:02.328 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[78092]: rados_ioctx_create. 2024-09-13T13:31:02.328 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[78092]: shutting down. 2024-09-13T13:31:02.328 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-13T13:31:02.328 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-09-13T13:31:02.328 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[77331]: starting. 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[77331]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-77228 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[77331]: created object 0... 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[77331]: created object 25... 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[77331]: created object 49... 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[77331]: finishing. 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[77331]: shutting down. 2024-09-13T13:31:02.338 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[77332]: starting. 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[77332]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-77228 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[77332]: shutting down. 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[78093]: starting. 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[78093]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-77228 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[78093]: created object 0... 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[78093]: created object 25... 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[78093]: created object 49... 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[78093]: finishing. 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[78093]: shutting down. 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[78095]: starting. 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[78095]: listing objects. 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[78095]: listed object 0... 2024-09-13T13:31:02.339 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[78095]: listed object 25... 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[78095]: saw 50 objects 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[78095]: shutting down. 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[78094]: starting. 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[78094]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-77228 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[78094]: shutting down. 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-09-13T13:31:02.340 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-13T13:31:02.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-13T13:31:02.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-13T13:31:02.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-13T13:31:02.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-13T13:31:02.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-13T13:31:02.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-13T13:31:02.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-13T13:31:02.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-13T13:31:02.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2495735397' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:02.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: osdmap e71: 8 total, 8 up, 8 in 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:02.750 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2495735397' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: osdmap e71: 8 total, 8 up, 8 in 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:02.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:02.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-13T13:31:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2495735397' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-13T13:31:02.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: osdmap e71: 8 total, 8 up, 8 in 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:02.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:02.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.362 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: Running main() from gmock_main.cc 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] Global test environment set-up. 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (1340 ms) 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (2001 ms) 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (3040 ms) 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] 3 tests from NeoRadosCmd (6381 ms total) 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] Global test environment tear-down 2024-09-13T13:31:03.363 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [==========] 3 tests from 1 test suite ran. (6381 ms total) 2024-09-13T13:31:03.364 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ PASSED ] 3 tests. 2024-09-13T13:31:03.406 INFO:tasks.workunit.client.0.smithi043.stdout:an_it": true} : dispatch 2024-09-13T13:31:03.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.380010+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-13T13:31:03.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.399643+0000 mon.b [INF] from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:03.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.400855+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:03.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.405595+0000 mon.b [INF] from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:03.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.407515+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.418294+0000 mon.b [INF] from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.427029+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.455012+0000 mon.c [INF] from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.455637+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.462338+0000 mon.c [INF] from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.463768+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.473174+0000 mon.c [INF] from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:01.475744+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304237+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304316+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-76803-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304385+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-76776-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304448+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-76354-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:03.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304519+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-77385-1"}]': finished 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304580+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304663+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304731+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304809+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-76417-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.304881+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.312744+0000 mon.b [INF] from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.313041+0000 mon.b [INF] from='client.? 172.21.15.43:0/3932460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.313192+0000 mon.b [INF] from='client.? 172.21.15.43:0/3012992646' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.313426+0000 mon.c [INF] from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.313697+0000 mon.c [INF] from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.318031+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:03.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.318640+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"} : dispatch 2024-09-13T13:31:03.456 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.322933+0000 mon.b [INF] from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.334732+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"} : dispatch 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.335852+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.338021+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.339102+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.390743+0000 mon.b [INF] from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:02.443528+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.309383+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:03.457 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.309480+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:03.458 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.309567+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:03.458 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.309652+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:03.458 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.309737+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:03.458 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.314877+0000 mon.b [INF] from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.458 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.315055+0000 mon.b [INF] from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:03.458 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.319077+0000 mon.c [INF] from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:03.460 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.334876+0000 mon.b [INF] from='client.? 172.21.15.43:0/431862295' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3"," api_io_pp: Running main() from gmock_main.cc 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] Running 40 tests from 2 test suites. 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: seed 76371 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (282 ms) 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (96 ms) 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-13T13:31:03.461 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (6 ms) 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (317 ms) 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (3 ms) 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (3 ms) 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (5 ms) 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (5 ms) 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-13T13:31:03.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (6 ms) 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (9 ms) 2024-09-13T13:31:03.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (3 ms) 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CrcZeroWrite 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CrcZeroWrite (4 ms) 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (4 ms) 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (1 ms) 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (3 ms) 2024-09-13T13:31:03.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP (764 ms total) 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1125 ms) 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (15 ms) 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-13T13:31:03.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (3 ms) 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.6 deep-scrub starts 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.6 deep-scrub ok 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.3 deep-scrub starts 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.3 deep-scrub ok 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.1 deep-scrub starts 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.1 deep-scrub ok 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.2 deep-scrub starts 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.2 deep-scrub ok 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: pgmap v54: 868 pgs: 480 unknown, 32 creating+peering, 356 active+clean; 587 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 417 op/s 2024-09-13T13:31:03.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.7 deep-scrub starts 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.7 deep-scrub ok 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.0 deep-scrub starts 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: 15.0 deep-scrub ok 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: osdmap e72: 8 total, 8 up, 8 in 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/431862295' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1372828117' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2507606386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:03.651 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: threexx 2024-09-13T13:31:03.652 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.6 deep-scrub starts 2024-09-13T13:31:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.6 deep-scrub ok 2024-09-13T13:31:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.3 deep-scrub starts 2024-09-13T13:31:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.3 deep-scrub ok 2024-09-13T13:31:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.1 deep-scrub starts 2024-09-13T13:31:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.1 deep-scrub ok 2024-09-13T13:31:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.2 deep-scrub starts 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.2 deep-scrub ok 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: pgmap v54: 868 pgs: 480 unknown, 32 creating+peering, 356 active+clean; 587 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 417 op/s 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.7 deep-scrub starts 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.7 deep-scrub ok 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.0 deep-scrub starts 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: 15.0 deep-scrub ok 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:03.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: osdmap e72: 8 total, 8 up, 8 in 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/431862295' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1372828117' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2507606386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: threexx 2024-09-13T13:31:03.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.6 deep-scrub starts 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.6 deep-scrub ok 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.3 deep-scrub starts 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.3 deep-scrub ok 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.1 deep-scrub starts 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.1 deep-scrub ok 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.2 deep-scrub starts 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.2 deep-scrub ok 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: pgmap v54: 868 pgs: 480 unknown, 32 creating+peering, 356 active+clean; 587 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 417 op/s 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.7 deep-scrub starts 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.7 deep-scrub ok 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.0 deep-scrub starts 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: 15.0 deep-scrub ok 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-76888-1"}]': finished 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-77444-1"}]': finished 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-76332-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-77385-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2462130284' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: osdmap e72: 8 total, 8 up, 8 in 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/431862295' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1372828117' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2507606386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: threexx 2024-09-13T13:31:03.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: 15.5 deep-scrub starts 2024-09-13T13:31:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: 15.5 deep-scrub ok 2024-09-13T13:31:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: 15.4 deep-scrub starts 2024-09-13T13:31:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: 15.4 deep-scrub ok 2024-09-13T13:31:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-13T13:31:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-13T13:31:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: fourxx 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3291975922' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: 15.8 deep-scrub starts 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: 15.8 deep-scrub ok 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: 15.5 deep-scrub starts 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: 15.5 deep-scrub ok 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: 15.4 deep-scrub starts 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: 15.4 deep-scrub ok 2024-09-13T13:31:04.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: fourxx 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3291975922' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: 15.8 deep-scrub starts 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: 15.8 deep-scrub ok 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:04.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: osdmap e73: 8 total, 8 up, 8 in 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:04.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: osdmap e73: 8 total, 8 up, 8 in 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"} : dispatch 2024-09-13T13:31:04.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:04.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:04.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:04.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: 15.5 deep-scrub starts 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: 15.5 deep-scrub ok 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: 15.4 deep-scrub starts 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: 15.4 deep-scrub ok 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: fourxx 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3291975922' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: 15.8 deep-scrub starts 2024-09-13T13:31:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: 15.8 deep-scrub ok 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-76417-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-76423-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi043-76371-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"} : dispatch 2024-09-13T13:31:04.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: osdmap e73: 8 total, 8 up, 8 in 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:04.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: ap list: Running main() from gmock_main.cc 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] Global test environment set-up. 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] 3 tests from NeoradosList 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjects (2341 ms) 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-09-13T13:31:05.342 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjectsNS (3012 ms) 2024-09-13T13:31:05.343 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-09-13T13:31:05.343 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjectsMany (3019 ms) 2024-09-13T13:31:05.343 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] 3 tests from NeoradosList (8372 ms total) 2024-09-13T13:31:05.343 INFO:tasks.workunit.client.0.smithi043.stdout: list: 2024-09-13T13:31:05.343 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] Global test environment tear-down 2024-09-13T13:31:05.343 INFO:tasks.workunit.client.0.smithi043.stdout: list: [==========] 3 tests from 1 test suite ran. (8372 ms total) 2024-09-13T13:31:05.343 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ PASSED ] 3 tests. 2024-09-13T13:31:05.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: 15.9 deep-scrub starts 2024-09-13T13:31:05.472 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: 15.9 deep-scrub starts 2024-09-13T13:31:05.472 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: 15.9 deep-scrub ok 2024-09-13T13:31:05.472 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: pgmap v57: 900 pgs: 512 unknown, 32 creating+peering, 356 active+clean; 587 KiB data, 263 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:05.472 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:05.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:31:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: 15.9 deep-scrub starts 2024-09-13T13:31:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: 15.9 deep-scrub ok 2024-09-13T13:31:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: pgmap v57: 900 pgs: 512 unknown, 32 creating+peering, 356 active+clean; 587 KiB data, 263 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"}]': finished 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:05.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:05.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:05.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: osdmap e74: 8 total, 8 up, 8 in 2024-09-13T13:31:05.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:05.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:05.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:05.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:31:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: 15.9 deep-scrub ok 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: pgmap v57: 900 pgs: 512 unknown, 32 creating+peering, 356 active+clean; 587 KiB data, 263 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"}]': finished 2024-09-13T13:31:05.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: osdmap e74: 8 total, 8 up, 8 in 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:05.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-4-cache", "mode": "writeback"}]': finished 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1480264294' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:05.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3297949683' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:05.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2543585964' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:05.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: osdmap e74: 8 total, 8 up, 8 in 2024-09-13T13:31:05.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"} : dispatch 2024-09-13T13:31:05.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"} : dispatch 2024-09-13T13:31:05.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"} : dispatch 2024-09-13T13:31:06.337 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Running main() from gmock_main.cc 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] Global test environment set-up. 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: handle_notify cookie 94782117110096 notify_id 287762808832 notifier_gid 24794 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (2262 ms) 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Trying... 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: handle_notify cookie 94782129813712 notify_id 300647710721 notifier_gid 24949 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Waiting for 3.000000000s 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Timed out. 2024-09-13T13:31:06.338 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Flushing... 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Flushed... 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7025 ms) 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (9287 ms total) 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] Global test environment tear-down 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (9287 ms total) 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ PASSED ] 2 tests. 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: Running main() from gmock_main.cc 2024-09-13T13:31:06.339 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] Global test environment set-up. 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (322 ms) 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-09-13T13:31:06.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (51 ms) 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (10 ms) 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.Checksum (7 ms) 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (4 ms) 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (6 ms) 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (4 ms) 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-13T13:31:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (5 ms) 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (5 ms) 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RmXattr (13 ms) 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrIter (7 ms) 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo (440 ms total) 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1114 ms) 2024-09-13T13:31:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (10 ms) 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (12 ms) 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (5 ms) 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-13T13:31:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (3 ms) 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1169 ms total) 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [---- api_stat_pp: Running main() from gmock_main.cc 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-13T13:31:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: seed 76803 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (264 ms) 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (15 ms) 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (13 ms) 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (47 ms) 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (339 ms total) 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-13T13:31:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1134 ms) 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2 ms) 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (8 ms) 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1144 ms total) 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (9597 ms total) 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: Running main() from gmock_main.cc 2024-09-13T13:31:06.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment set-up. 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.Stat (286 ms) 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (49 ms) 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-13T13:31:06.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.StatNS (13 ms) 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (4 ms) 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 5 tests from LibRadosStat (353 ms total) 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1127 ms) 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (9 ms) 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-09-13T13:31:06.348 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (9 ms) 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1146 ms total) 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment tear-down 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (9604 ms total) 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ PASSED ] 9 tests. 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout:------] Global test environment tear-down 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] 24 tests from 2 test suites ran. (9769 ms total) 2024-09-13T13:31:06.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ PASSED ] 24 tests. 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout:app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.338608+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-4", "overlaypool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.338844+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-76332-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.339583+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi043-77385-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.340486+0000 mon.c [INF] from='client.? 172.21.15.43:0/1372828117' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.340853+0000 mon.c [INF] from='client.? 172.21.15.43:0/2507606386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.341646+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-76315-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.477 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.342173+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.342546+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.354863+0000 mon.c [INF] from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.355610+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-77444-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.399798+0000 mon.b [INF] from='client.? 172.21.15.43:0/862481980' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.399859+0000 client.admin [INF] threexx 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-09-13T13:31:03.417048+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7141 ms) 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (9704 ms total) 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (9705 ms total) 2024-09-13T13:31:06.478 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-13T13:31:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4} : dispatch 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4} : dispatch 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"} : dispatch 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"} : dispatch 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4}]': finished 2024-09-13T13:31:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"}]': finished 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: osdmap e75: 8 total, 8 up, 8 in 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2998727219' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3324496102' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4} : dispatch 2024-09-13T13:31:06.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4} : dispatch 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"} : dispatch 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"} : dispatch 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4}]': finished 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"}]': finished 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: osdmap e75: 8 total, 8 up, 8 in 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:06.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2998727219' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3324496102' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3212660762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3212660762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4} : dispatch 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4} : dispatch 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"} : dispatch 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"} : dispatch 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-77444-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-76776-7"}]': finished 2024-09-13T13:31:06.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-76803-7"}]': finished 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-76354-16"}]': finished 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76888-6", "pg_num": 4}]': finished 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-4"}]': finished 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: osdmap e75: 8 total, 8 up, 8 in 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1831787112' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2998727219' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"} : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3324496102' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3212660762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:06.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: pgmap v60: 740 pgs: 80 creating+peering, 40 unknown, 620 active+clean; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 145 op/s 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: pool 'PoolQuotaPP_smithi043-76332-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3324496102' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: osdmap e76: 8 total, 8 up, 8 in 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: pgmap v60: 740 pgs: 80 creating+peering, 40 unknown, 620 active+clean; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 145 op/s 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: pool 'PoolQuotaPP_smithi043-76332-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3324496102' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: osdmap e76: 8 total, 8 up, 8 in 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:07.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: pgmap v60: 740 pgs: 80 creating+peering, 40 unknown, 620 active+clean; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 145 op/s 2024-09-13T13:31:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: pool 'PoolQuotaPP_smithi043-76332-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-4", "tierpool": "test-rados-api-smithi043-76627-4-cache"}]': finished 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3324496102' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-76315-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: osdmap e76: 8 total, 8 up, 8 in 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:07.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:07.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:07.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:08.858 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: Running main() from gmock_main.cc 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] Global test environment set-up. 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (5351 ms) 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (6539 ms) 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (11890 ms total) 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] Global test environment tear-down 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (11890 ms total) 2024-09-13T13:31:08.859 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ PASSED ] 2 tests. 2024-09-13T13:31:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"}]': finished 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: osdmap e77: 8 total, 8 up, 8 in 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2808493328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:09.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: pgmap v64: 552 pgs: 32 creating+peering, 132 unknown, 388 active+clean; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 39 MiB/s wr, 133 op/s 2024-09-13T13:31:09.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"}]': finished 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: osdmap e77: 8 total, 8 up, 8 in 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"} : dispatch 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2808493328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: pgmap v64: 552 pgs: 32 creating+peering, 132 unknown, 388 active+clean; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 39 MiB/s wr, 133 op/s 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:09.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "overlaypool": "test-rados-api-smithi043-76888-6"}]': finished 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: osdmap e77: 8 total, 8 up, 8 in 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3717363396' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2808493328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3780537731' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:09.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:09.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: pgmap v64: 552 pgs: 32 creating+peering, 132 unknown, 388 active+clean; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 39 MiB/s wr, 133 op/s 2024-09-13T13:31:09.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:09.864 INFO:tasks.workunit.client.0.smithi043.stdout: pool: Running main() from gmock_main.cc 2024-09-13T13:31:09.864 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] Global test environment set-up. 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolList (1299 ms) 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (1983 ms) 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2009 ms) 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4015 ms) 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (1511 ms) 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-09-13T13:31:09.865 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2017 ms) 2024-09-13T13:31:09.866 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] 6 tests from NeoRadosPools (12834 ms total) 2024-09-13T13:31:09.866 INFO:tasks.workunit.client.0.smithi043.stdout: pool: 2024-09-13T13:31:09.866 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] Global test environment tear-down 2024-09-13T13:31:09.866 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [==========] 6 tests from 1 test suite ran. (12834 ms total) 2024-09-13T13:31:09.866 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ PASSED ] 6 tests. 2024-09-13T13:31:09.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-13T13:31:09.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-13T13:31:09.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-13T13:31:09.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-13T13:31:09.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: seed 76423 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (377 ms) 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (6 ms) 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1009 ms) 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1019 ms) 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (4 ms) 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-13T13:31:09.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (6 ms) 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2429 ms total) 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1144 ms) 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (9 ms) 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1042 ms) 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-09-13T13:31:09.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3216 ms total) 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (13266 ms total) 2024-09-13T13:31:09.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: Running main() from gmock_main.cc 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment set-up. 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (336 ms) 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockShared (48 ms) 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1007 ms) 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1019 ms) 2024-09-13T13:31:09.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.Unlock (6 ms) 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (5 ms) 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock (2428 ms total) 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1120 ms) 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-13T13:31:09.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (14 ms) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1066 ms) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1004 ms) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (5 ms) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3221 ms total) 2024-09-13T13:31:09.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-09-13T13:31:09.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment tear-down 2024-09-13T13:31:09.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (13274 ms total) 2024-09-13T13:31:09.875 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-13T13:31:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: osdmap e78: 8 total, 8 up, 8 in 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2856741918' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3839594466' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/532386257' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.151 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"}]': finished 2024-09-13T13:31:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: osdmap e78: 8 total, 8 up, 8 in 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2856741918' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3839594466' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/532386257' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:10.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76888-6", "mode": "writeback"}]': finished 2024-09-13T13:31:10.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:10.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:10.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-77385-2"}]': finished 2024-09-13T13:31:10.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-76650-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:10.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: osdmap e78: 8 total, 8 up, 8 in 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1673363616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2410688581' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1124857102' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2856741918' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"} : dispatch 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"} : dispatch 2024-09-13T13:31:10.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3839594466' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/532386257' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-13T13:31:10.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.248 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:10.248 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:11.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3839594466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/532386257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2"} : dispatch 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: osdmap e79: 8 total, 8 up, 8 in 2024-09-13T13:31:11.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:11.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:11.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-77444-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:11.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: pgmap v67: 648 pgs: 32 creating+peering, 616 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:11.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3839594466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/532386257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2"} : dispatch 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: osdmap e79: 8 total, 8 up, 8 in 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-77444-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: pgmap v67: 648 pgs: 32 creating+peering, 616 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-76417-10"}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-76423-10"}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3839594466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/532386257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-77444-2"}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-76315-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2"} : dispatch 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: osdmap e79: 8 total, 8 up, 8 in 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:11.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-77444-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:11.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: pgmap v67: 648 pgs: 32 creating+peering, 616 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:11.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:11.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-13T13:31:11.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-13T13:31:11.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-13T13:31:11.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1118 ms) 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1122 ms total) 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (152 ms) 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-13T13:31:11.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3442 ms) 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (11 ms) 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (14 ms) 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630443744 notify_id 335007449090 notifier_gid 25169 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630443744 notify_id 335007449091 notifier_gid 25169 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630443744 notify_id 335007449092 notifier_gid 25169 2024-09-13T13:31:11.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (4 ms) 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630459360 notify_id 335007449092 notifier_gid 25169 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630443744 notify_id 335007449093 notifier_gid 25169 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630459360 notify_id 335007449090 notifier_gid 25169 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630459360 notify_id 335007449091 notifier_gid 25169 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-09-13T13:31:11.983 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-77444-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: osdmap e80: 8 total, 8 up, 8 in 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-77444-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[40255]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-77444-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: osdmap e80: 8 total, 8 up, 8 in 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-77444-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:12.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:12.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:12.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:11 smithi043 ceph-mon[30106]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-76844-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-76458-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-77444-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-13T13:31:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: osdmap e80: 8 total, 8 up, 8 in 2024-09-13T13:31:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:31:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-77444-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:31:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:11 smithi189 ceph-mon[32912]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: osdmap e81: 8 total, 8 up, 8 in 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/675735367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1251429806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2716909683' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/883219691' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: pgmap v70: 696 pgs: 304 unknown, 392 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:31:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-77444-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: osdmap e81: 8 total, 8 up, 8 in 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/675735367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1251429806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2716909683' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/883219691' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: pgmap v70: 696 pgs: 304 unknown, 392 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-77444-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: osdmap e82: 8 total, 8 up, 8 in 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-13T13:31:13.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: osdmap e82: 8 total, 8 up, 8 in 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:31:13.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:31:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-76371-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-13T13:31:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: osdmap e81: 8 total, 8 up, 8 in 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/675735367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1251429806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2716909683' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/883219691' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: pgmap v70: 696 pgs: 304 unknown, 392 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-77444-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-76709-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-76315-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: osdmap e82: 8 total, 8 up, 8 in 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:13.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2573269762' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:31:13.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:31:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"}]': finished 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: osdmap e83: 8 total, 8 up, 8 in 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"}]': finished 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: osdmap e83: 8 total, 8 up, 8 in 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-76458-1","app":"app1","key":"key1"}]': finished 2024-09-13T13:31:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: osdmap e83: 8 total, 8 up, 8 in 2024-09-13T13:31:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/486729463' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"} : dispatch 2024-09-13T13:31:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_wa api_watch_notify: Running main() from gmock_main.cc 2024-09-13T13:31:14.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-13T13:31:14.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-13T13:31:14.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-13T13:31:14.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (370 ms) 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94010604702848 err -107 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (6 ms) 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94010604721136 err -107 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (12 ms) 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-13T13:31:14.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 287762808832 cookie 94010604736656 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (12 ms) 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 287762808833 cookie 94010604736656 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (12 ms) 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 287762808833 cookie 94010604736656 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (7 ms) 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 287762808834 cookie 94010604736656 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 287762808834 cookie 94010604775920 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (10 ms) 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-13T13:31:14.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 287762808833 cookie 94010604775920 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 292057776130 cookie 94010604775920 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3010 ms) 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94010604775920 err -107 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24697 notify_id 322122547201 cookie 94010604770656 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94010604770656 err -107 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9451 ms total) 2024-09-13T13:31:14.923 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1079 ms) 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1079 ms total) 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-13T13:31:14.924 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (18163 ms total) 2024-09-13T13:31:14.936 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-13T13:31:14.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:14.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:14.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: pgmap v73: 496 pgs: 168 unknown, 328 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:14.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: osdmap e84: 8 total, 8 up, 8 in 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1887397819' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1441709274' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/920837003' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:14.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:14.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: pgmap v73: 496 pgs: 168 unknown, 328 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: osdmap e84: 8 total, 8 up, 8 in 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1887397819' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1441709274' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/920837003' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.269 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:15.269 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:15.269 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: pgmap v73: 496 pgs: 168 unknown, 328 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:15.269 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:15.269 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-76844-12"}]': finished 2024-09-13T13:31:15.269 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: osdmap e84: 8 total, 8 up, 8 in 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1887397819' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1441709274' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3407129187' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/920837003' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"} : dispatch 2024-09-13T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:15.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:31:15.917 INFO:tasks.workunit.client.0.smithi043.stdout:i_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-13T13:31:15.917 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-09-13T13:31:15.917 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-13T13:31:15.917 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (4 ms) 2024-09-13T13:31:15.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-13T13:31:15.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-09-13T13:31:15.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-13T13:31:15.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (4 ms) 2024-09-13T13:31:15.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-13T13:31:15.918 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (2 ms) 2024-09-13T13:31:15.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-13T13:31:15.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-09-13T13:31:15.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-13T13:31:15.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (3 ms) 2024-09-13T13:31:15.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-13T13:31:15.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-09-13T13:31:15.919 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-13T13:31:15.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-09-13T13:31:15.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-13T13:31:15.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-09-13T13:31:15.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CrcZeroWrite 2024-09-13T13:31:15.920 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CrcZeroWrite (8377 ms) 2024-09-13T13:31:15.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-13T13:31:15.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (1108 ms) 2024-09-13T13:31:15.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-13T13:31:15.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (10 ms) 2024-09-13T13:31:15.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-13T13:31:15.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-09-13T13:31:15.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-13T13:31:15.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (9 ms) 2024-09-13T13:31:15.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP (10690 ms total) 2024-09-13T13:31:15.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-09-13T13:31:15.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-13T13:31:15.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] 40 tests from 2 test suites ran. (19338 ms total) 2024-09-13T13:31:15.922 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ PASSED ] 40 tests. 2024-09-13T13:31:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:31:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"}]': finished 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: osdmap e85: 8 total, 8 up, 8 in 2024-09-13T13:31:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"}]': finished 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: osdmap e85: 8 total, 8 up, 8 in 2024-09-13T13:31:15.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-77055-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-7"}]': finished 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-76315-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-76371-24"}]': finished 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: osdmap e85: 8 total, 8 up, 8 in 2024-09-13T13:31:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:17.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: pgmap v76: 680 pgs: 192 creating+peering, 64 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: osdmap e86: 8 total, 8 up, 8 in 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3116345116' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: pgmap v76: 680 pgs: 192 creating+peering, 64 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: osdmap e86: 8 total, 8 up, 8 in 2024-09-13T13:31:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:17.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3116345116' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:17.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:17.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:17.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: pgmap v76: 680 pgs: 192 creating+peering, 64 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-13T13:31:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: osdmap e86: 8 total, 8 up, 8 in 2024-09-13T13:31:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-77444-3"} : dispatch 2024-09-13T13:31:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3116345116' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"} : dispatch 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: Running main() from gmock_main.cc 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] Global test environment set-up. 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (6338 ms) 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (6552 ms) 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (8028 ms) 2024-09-13T13:31:17.905 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] 3 tests from NeoradosECList (20918 ms total) 2024-09-13T13:31:17.906 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: 2024-09-13T13:31:17.906 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] Global test environment tear-down 2024-09-13T13:31:17.906 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (20920 ms total) 2024-09-13T13:31:17.906 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ PASSED ] 3 tests. 2024-09-13T13:31:18.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"}]': finished 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: osdmap e87: 8 total, 8 up, 8 in 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/255085607' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1030274433' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"}]': finished 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: osdmap e87: 8 total, 8 up, 8 in 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/255085607' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1030274433' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1491422672' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-77444-3"}]': finished 2024-09-13T13:31:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-7"}]': finished 2024-09-13T13:31:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: osdmap e87: 8 total, 8 up, 8 in 2024-09-13T13:31:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/255085607' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1030274433' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[40255]: pgmap v79: 680 pgs: 64 creating+peering, 192 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-13T13:31:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[40255]: osdmap e88: 8 total, 8 up, 8 in 2024-09-13T13:31:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[30106]: pgmap v79: 680 pgs: 64 creating+peering, 192 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-13T13:31:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:18 smithi043 ceph-mon[30106]: osdmap e88: 8 total, 8 up, 8 in 2024-09-13T13:31:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:18 smithi189 ceph-mon[32912]: pgmap v79: 680 pgs: 64 creating+peering, 192 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-13T13:31:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-76315-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-76332-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:18 smithi189 ceph-mon[32912]: osdmap e88: 8 total, 8 up, 8 in 2024-09-13T13:31:19.949 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: Running main() from gmock_main.cc 2024-09-13T13:31:19.949 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-09-13T13:31:19.949 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] Global test environment set-up. 2024-09-13T13:31:19.949 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-09-13T13:31:19.949 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (2270 ms) 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3001 ms) 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (3037 ms) 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (2480 ms) 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3043 ms) 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3005 ms) 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-09-13T13:31:19.950 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (6054 ms) 2024-09-13T13:31:19.951 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (22890 ms total) 2024-09-13T13:31:19.951 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: 2024-09-13T13:31:19.951 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] Global test environment tear-down 2024-09-13T13:31:19.951 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (22890 ms total) 2024-09-13T13:31:19.951 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ PASSED ] 7 tests. 2024-09-13T13:31:19.959 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-13T13:31:19.960 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (2490 ms) 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3015 ms) 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3004 ms) 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2507 ms) 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3046 ms) 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3008 ms) 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-13T13:31:19.961 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (6058 ms) 2024-09-13T13:31:19.962 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (23129 ms total) 2024-09-13T13:31:19.962 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-09-13T13:31:19.962 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-13T13:31:19.962 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (23130 ms total) 2024-09-13T13:31:19.962 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-13T13:31:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:20 smithi189 ceph-mon[32912]: osdmap e89: 8 total, 8 up, 8 in 2024-09-13T13:31:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:20.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[30106]: osdmap e89: 8 total, 8 up, 8 in 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:20.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[40255]: osdmap e89: 8 total, 8 up, 8 in 2024-09-13T13:31:20.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:20.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: pgmap v82: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-13T13:31:21.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:21.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:21.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: osdmap e90: 8 total, 8 up, 8 in 2024-09-13T13:31:21.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2846179161' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.302 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.302 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2386920517' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.302 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.302 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"} : dispatch 2024-09-13T13:31:21.302 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"} : dispatch 2024-09-13T13:31:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: pgmap v82: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: osdmap e90: 8 total, 8 up, 8 in 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2846179161' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2386920517' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"} : dispatch 2024-09-13T13:31:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"} : dispatch 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: pgmap v82: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: osdmap e90: 8 total, 8 up, 8 in 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2846179161' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2386920517' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"} : dispatch 2024-09-13T13:31:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"} : dispatch 2024-09-13T13:31:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"}]': finished 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: osdmap e91: 8 total, 8 up, 8 in 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2026091850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"}]': finished 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: osdmap e91: 8 total, 8 up, 8 in 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:22.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2026091850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:22.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:22.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:22.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:22.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:22.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:22.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-76332-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-76315-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4"}]': finished 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: osdmap e91: 8 total, 8 up, 8 in 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"} : dispatch 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2026091850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:22.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: pgmap v85: 616 pgs: 192 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"}]': finished 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"}]': finished 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: osdmap e92: 8 total, 8 up, 8 in 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"} : dispatch 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"} : dispatch 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:23.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: pgmap v85: 616 pgs: 192 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"}]': finished 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"}]': finished 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: osdmap e92: 8 total, 8 up, 8 in 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:31:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: pgmap v85: 616 pgs: 192 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-13T13:31:23.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-76888-4", "tierpool": "test-rados-api-smithi043-76888-6"}]': finished 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-9"}]': finished 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: osdmap e92: 8 total, 8 up, 8 in 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074670459' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"} : dispatch 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"} : dispatch 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:23.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:23.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment set-up. 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5067 ms) 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterLate (15 ms) 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-09-13T13:31:24.130 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_WARN 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 10 pool(s) do not have an application enabled 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 87s), standbys: x 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 24 pools, 636 pgs 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 403 objects, 609 KiB 2024-09-13T13:31:24.131 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 263 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 23.899% pgs not active 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 484 active+clean 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 152 creating+peering 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 3.2 KiB/s rd, 90 KiB/s wr, 130 op/s rd, 243 op/s wr 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_WARN 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 12 pool(s) do not have an application enabled 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.132 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 89s), standbys: x 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 29 pools, 868 pgs 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 244 objects, 587 KiB 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 263 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 55.300% pgs unknown 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 3.687% pgs not active 2024-09-13T13:31:24.133 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 480 unknown 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 356 active+clean 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 32 creating+peering 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 3.0 KiB/s rd, 22 KiB/s wr, 192 op/s rd, 225 op/s wr 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.StatusFormat (2178 ms) 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.Status (20038 ms) 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService (27299 ms total) 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment tear-down 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [==========] 4 tests from 1 test suite ran. (27299 ms total) 2024-09-13T13:31:24.134 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ PASSED ] 4 tests. 2024-09-13T13:31:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:31:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"}]': finished 2024-09-13T13:31:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: osdmap e93: 8 total, 8 up, 8 in 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3658909601' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/979854831' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi043-76315-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:31:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"}]': finished 2024-09-13T13:31:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: osdmap e93: 8 total, 8 up, 8 in 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3658909601' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/979854831' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi043-76315-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:24.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-76888-6", "pool2": "test-rados-api-smithi043-76888-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-9", "mode": "writeback"}]': finished 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: osdmap e93: 8 total, 8 up, 8 in 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3658909601' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/979854831' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi043-76315-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:24.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:24.925 INFO:tasks.workunit.client.0.smithi043.stdout:tch_notify_pp: flushed 2024-09-13T13:31:24.926 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-09-13T13:31:24.926 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-13T13:31:24.926 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-09-13T13:31:24.926 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630459328 notify_id 347892350980 notifier_gid 25169 2024-09-13T13:31:24.927 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-09-13T13:31:24.927 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-13T13:31:24.927 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushed 2024-09-13T13:31:24.927 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3004 ms) 2024-09-13T13:31:24.927 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-13T13:31:24.927 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630477312 notify_id 360777252870 notifier_gid 25169 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-09-13T13:31:24.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3012 ms) 2024-09-13T13:31:24.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-13T13:31:24.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-09-13T13:31:24.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-09-13T13:31:24.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94382630477312 notify_id 373662154757 notifier_gid 25169 2024-09-13T13:31:24.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-09-13T13:31:24.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-09-13T13:31:24.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3010 ms) 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (15678 ms total) 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (28150 ms total) 2024-09-13T13:31:24.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-13T13:31:25.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:31:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: pgmap v88: 612 pgs: 224 unknown, 32 creating+peering, 356 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/979854831' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-76315-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: osdmap e94: 8 total, 8 up, 8 in 2024-09-13T13:31:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: pgmap v88: 612 pgs: 224 unknown, 32 creating+peering, 356 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/979854831' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-76315-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: osdmap e94: 8 total, 8 up, 8 in 2024-09-13T13:31:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:25.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: pgmap v88: 612 pgs: 224 unknown, 32 creating+peering, 356 active+clean; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:31:25.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:25.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/979854831' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-76315-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:25.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-76332-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:25.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:25.497 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:25.497 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: osdmap e94: 8 total, 8 up, 8 in 2024-09-13T13:31:25.497 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"} : dispatch 2024-09-13T13:31:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:31:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:31:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:31:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:26 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"}]': finished 2024-09-13T13:31:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:26 smithi189 ceph-mon[32912]: osdmap e95: 8 total, 8 up, 8 in 2024-09-13T13:31:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"}]': finished 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[30106]: osdmap e95: 8 total, 8 up, 8 in 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-76650-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-9"}]': finished 2024-09-13T13:31:26.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:26 smithi043 ceph-mon[40255]: osdmap e95: 8 total, 8 up, 8 in 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5050 ms) 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (48 ms) 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-13T13:31:27.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20043 ms) 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 0 of 20 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: failed to deregister: 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: { 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "epoch": 19, 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "modified": "2024-09-13T13:31:24.198256+0000", 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "services": { 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "laundry": { 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "close-test-pid76968": { 2024-09-13T13:31:27.116 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 18, 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-13T13:31:22.036578+0000", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 30821, 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/926052162", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "squid", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa43119222b) squid (dev)", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "19.3.0-4917-gb0201b8c", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-13T13:31:27.117 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493480", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-13T13:31:27.118 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "rgw": { 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "24361": { 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 12, 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-09-13T13:29:09.812515+0000", 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24361, 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/3531949875", 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-09-13T13:31:27.119 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "squid", 2024-09-13T13:31:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: pgmap v91: 396 pgs: 8 unknown, 388 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: osdmap e96: 8 total, 8 up, 8 in 2024-09-13T13:31:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3113808072' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/629308734' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-76332-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3115935502' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-13T13:31:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: pgmap v91: 396 pgs: 8 unknown, 388 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: osdmap e96: 8 total, 8 up, 8 in 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3113808072' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/629308734' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-76332-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3115935502' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-13T13:31:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: pgmap v91: 396 pgs: 8 unknown, 388 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: osdmap e96: 8 total, 8 up, 8 in 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3113808072' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/629308734' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-76332-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:27.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3115935502' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-13T13:31:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-13T13:31:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/629308734' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-76332-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: osdmap e97: 8 total, 8 up, 8 in 2024-09-13T13:31:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-13T13:31:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/629308734' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-76332-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: osdmap e97: 8 total, 8 up, 8 in 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76623-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-76315-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/629308734' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-76332-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:28.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: osdmap e97: 8 total, 8 up, 8 in 2024-09-13T13:31:28.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:28.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:28.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 19.3.0-4917-gb0201b8c (b0201b8c79733293453e7f10a10c7fa431 misc: Running main() from gmock_main.cc 2024-09-13T13:31:28.949 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-09-13T13:31:28.949 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] Global test environment set-up. 2024-09-13T13:31:28.949 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Version (1293 ms) 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (1998 ms) 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongName (3029 ms) 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (2992 ms) 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2519 ms) 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3033 ms) 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-09-13T13:31:28.950 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Exec (3023 ms) 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3021 ms) 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (2995 ms) 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2969 ms) 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1989 ms) 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3039 ms) 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] 12 tests from NeoRadosMisc (31900 ms total) 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] Global test environment tear-down 2024-09-13T13:31:28.951 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [==========] 12 tests from 1 test suite ran. (31900 ms total) 2024-09-13T13:31:28.952 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ PASSED ] 12 tests. 2024-09-13T13:31:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:29 smithi189 ceph-mon[32912]: pgmap v94: 588 pgs: 264 unknown, 324 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-09-13T13:31:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:29 smithi189 ceph-mon[32912]: osdmap e98: 8 total, 8 up, 8 in 2024-09-13T13:31:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""} : dispatch 2024-09-13T13:31:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""} : dispatch 2024-09-13T13:31:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[30106]: pgmap v94: 588 pgs: 264 unknown, 324 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-09-13T13:31:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[30106]: osdmap e98: 8 total, 8 up, 8 in 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""} : dispatch 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""} : dispatch 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[40255]: pgmap v94: 588 pgs: 264 unknown, 324 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[40255]: osdmap e98: 8 total, 8 up, 8 in 2024-09-13T13:31:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""} : dispatch 2024-09-13T13:31:29.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""} : dispatch 2024-09-13T13:31:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""}]': finished 2024-09-13T13:31:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: osdmap e99: 8 total, 8 up, 8 in 2024-09-13T13:31:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/980287510' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/428314613' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:30.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""}]': finished 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: osdmap e99: 8 total, 8 up, 8 in 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/980287510' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/428314613' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache", "force_nonempty":""}]': finished 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: osdmap e99: 8 total, 8 up, 8 in 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/980287510' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/428314613' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:30.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:30.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: pgmap v97: 588 pgs: 128 creating+peering, 460 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: osdmap e100: 8 total, 8 up, 8 in 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"} : dispatch 2024-09-13T13:31:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"} : dispatch 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: pgmap v97: 588 pgs: 128 creating+peering, 460 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:31.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: osdmap e100: 8 total, 8 up, 8 in 2024-09-13T13:31:31.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"} : dispatch 2024-09-13T13:31:31.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:31.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"} : dispatch 2024-09-13T13:31:31.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: pgmap v97: 588 pgs: 128 creating+peering, 460 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-76315-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76623-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: osdmap e100: 8 total, 8 up, 8 in 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3473742689' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"} : dispatch 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"} : dispatch 2024-09-13T13:31:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:32.240 INFO:tasks.workunit.client.0.smithi043.stdout:19222b) squid (dev)", 2024-09-13T13:31:32.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "19.3.0-4917-gb0201b8c", 2024-09-13T13:31:32.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_hostname": "smithi043", 2024-09-13T13:31:32.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b", 2024-09-13T13:31:32.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-13T13:31:32.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_type#0": "beast", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "id": "foo.a", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32493480", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "num_handles": "1", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-09-13T13:31:32.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid": "7", 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_id": "", 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_name": "", 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_id": "3e36b5e1-64e2-4c70-8cce-370c1604a33a", 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_name": "default", 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_id": "35f318ea-1fe4-408e-842f-db87241d19df", 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_name": "default" 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-13T13:31:32.242 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 1 of 20 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10255 ms) 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (35396 ms total) 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (35396 ms total) 2024-09-13T13:31:32.243 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-13T13:31:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-13T13:31:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"}]': finished 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"}]': finished 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: osdmap e101: 8 total, 8 up, 8 in 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"} : dispatch 2024-09-13T13:31:32.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"} : dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-13T13:31:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-13T13:31:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"}]': finished 2024-09-13T13:31:32.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"}]': finished 2024-09-13T13:31:32.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: osdmap e101: 8 total, 8 up, 8 in 2024-09-13T13:31:32.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"} : dispatch 2024-09-13T13:31:32.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"} : dispatch 2024-09-13T13:31:32.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:32.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-13T13:31:32.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-13T13:31:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76623-10", "tierpool":"test-rados-api-smithi043-76623-10-cache"}]': finished 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-11"}]': finished 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: osdmap e101: 8 total, 8 up, 8 in 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"} : dispatch 2024-09-13T13:31:32.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"} : dispatch 2024-09-13T13:31:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.6 deep-scrub starts 2024-09-13T13:31:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.6 deep-scrub ok 2024-09-13T13:31:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.0 deep-scrub starts 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.0 deep-scrub ok 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.4 deep-scrub starts 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.4 deep-scrub ok 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.1 deep-scrub starts 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.3 deep-scrub starts 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.3 deep-scrub ok 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: pgmap v100: 460 pgs: 460 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 259 KiB/s wr, 7 op/s 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.1 deep-scrub ok 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3077981067' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:33.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"}]': finished 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: osdmap e102: 8 total, 8 up, 8 in 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1933405673' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/883947874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.7 deep-scrub starts 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[30106]: 15.7 deep-scrub ok 2024-09-13T13:31:33.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.6 deep-scrub starts 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.6 deep-scrub ok 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.0 deep-scrub starts 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.0 deep-scrub ok 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.4 deep-scrub starts 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.4 deep-scrub ok 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.1 deep-scrub starts 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.3 deep-scrub starts 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.3 deep-scrub ok 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: pgmap v100: 460 pgs: 460 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 259 KiB/s wr, 7 op/s 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.1 deep-scrub ok 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3077981067' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:33.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"}]': finished 2024-09-13T13:31:33.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: osdmap e102: 8 total, 8 up, 8 in 2024-09-13T13:31:33.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1933405673' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/883947874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.7 deep-scrub starts 2024-09-13T13:31:33.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:33 smithi043 ceph-mon[40255]: 15.7 deep-scrub ok 2024-09-13T13:31:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.6 deep-scrub starts 2024-09-13T13:31:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.6 deep-scrub ok 2024-09-13T13:31:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.0 deep-scrub starts 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.0 deep-scrub ok 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.4 deep-scrub starts 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.4 deep-scrub ok 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.1 deep-scrub starts 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.3 deep-scrub starts 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.3 deep-scrub ok 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: pgmap v100: 460 pgs: 460 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 259 KiB/s wr, 7 op/s 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.1 deep-scrub ok 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3077981067' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-11", "mode": "writeback"}]': finished 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: osdmap e102: 8 total, 8 up, 8 in 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1933405673' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/883947874' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.7 deep-scrub starts 2024-09-13T13:31:33.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:33 smithi189 ceph-mon[32912]: 15.7 deep-scrub ok 2024-09-13T13:31:33.872 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: Running main() from gmock_main.cc 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment set-up. 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolList (2649 ms) 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3002 ms) 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3011 ms) 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2543 ms) 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-13T13:31:33.873 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3024 ms) 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5008 ms) 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5034 ms) 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4934 ms) 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7986 ms) 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools (37191 ms total) 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment tear-down 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (37191 ms total) 2024-09-13T13:31:33.874 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: 15.2 deep-scrub starts 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: 15.2 deep-scrub ok 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: 15.5 deep-scrub starts 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: 15.5 deep-scrub ok 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"}]': finished 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: osdmap e103: 8 total, 8 up, 8 in 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:34.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: 15.2 deep-scrub starts 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: 15.2 deep-scrub ok 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: 15.5 deep-scrub starts 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: 15.5 deep-scrub ok 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"}]': finished 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: osdmap e103: 8 total, 8 up, 8 in 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:34.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:34.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:34.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: 15.2 deep-scrub starts 2024-09-13T13:31:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: 15.2 deep-scrub ok 2024-09-13T13:31:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: 15.5 deep-scrub starts 2024-09-13T13:31:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: 15.5 deep-scrub ok 2024-09-13T13:31:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-76332-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-76315-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"}]': finished 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: osdmap e103: 8 total, 8 up, 8 in 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:31:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: 15.8 deep-scrub starts 2024-09-13T13:31:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: 15.8 deep-scrub ok 2024-09-13T13:31:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: 15.9 deep-scrub starts 2024-09-13T13:31:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: 15.9 deep-scrub ok 2024-09-13T13:31:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: pgmap v103: 524 pgs: 160 unknown, 364 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"}]': finished 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: osdmap e104: 8 total, 8 up, 8 in 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: 15.8 deep-scrub starts 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: 15.8 deep-scrub ok 2024-09-13T13:31:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: 15.9 deep-scrub starts 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: 15.9 deep-scrub ok 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: pgmap v103: 524 pgs: 160 unknown, 364 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"}]': finished 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: osdmap e104: 8 total, 8 up, 8 in 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:35.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:35.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:31:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: 15.8 deep-scrub starts 2024-09-13T13:31:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: 15.8 deep-scrub ok 2024-09-13T13:31:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: 15.9 deep-scrub starts 2024-09-13T13:31:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: 15.9 deep-scrub ok 2024-09-13T13:31:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: pgmap v103: 524 pgs: 160 unknown, 364 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"}]': finished 2024-09-13T13:31:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:31:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: osdmap e104: 8 total, 8 up, 8 in 2024-09-13T13:31:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"} : dispatch 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment set-up. 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-13T13:31:35.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-13T13:30:56.702+0000 7fddd99f8780 -1 failed for service _ceph-mon._tcp 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-13T13:30:56.702+0000 7fddd99f8780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (65 ms) 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5017 ms) 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5082 ms total) 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x7fddc006a3a0 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55625b06a150 2024-09-13T13:31:35.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 2 aios 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x7fddc006a3a0 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55625b06a150 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: done. 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (5119 ms) 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (5119 ms total) 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Exec (83 ms) 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-13T13:31:35.889 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (5 ms) 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (2 ms) 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4935 ms) 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (21658 ms) 2024-09-13T13:31:35.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc (26684 ms total) 2024-09-13T13:31:35.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-09-13T13:31:35.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment tear-down 2024-09-13T13:31:35.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (39228 ms total) 2024-09-13T13:31:35.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-13T13:31:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:31:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:31:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"}]': finished 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: osdmap e105: 8 total, 8 up, 8 in 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4152654010' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-76315-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1008199805' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"}]': finished 2024-09-13T13:31:36.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: osdmap e105: 8 total, 8 up, 8 in 2024-09-13T13:31:36.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4152654010' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-76315-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1008199805' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:36.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:36.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-11"}]': finished 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: osdmap e105: 8 total, 8 up, 8 in 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4152654010' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-76315-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1008199805' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:36.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: pgmap v106: 484 pgs: 128 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 11 op/s 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4152654010' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-76315-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: osdmap e106: 8 total, 8 up, 8 in 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: pgmap v106: 484 pgs: 128 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 11 op/s 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4152654010' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-76315-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: osdmap e106: 8 total, 8 up, 8 in 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: pgmap v106: 484 pgs: 128 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 11 op/s 2024-09-13T13:31:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4152654010' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-76315-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-76332-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:37.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: osdmap e106: 8 total, 8 up, 8 in 2024-09-13T13:31:37.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3013786244' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:37.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"} : dispatch 2024-09-13T13:31:37.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:37.875 INFO:tasks.workunit.client.0.smithi043.stdout: io: Running main() from gmock_main.cc 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] Global test environment set-up. 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] 14 tests from NeoRadosIo 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Limits (2336 ms) 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (3009 ms) 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.ReadOp (3017 ms) 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.SparseRead (2510 ms) 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-09-13T13:31:37.876 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3036 ms) 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3005 ms) 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3012 ms) 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3032 ms) 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2947 ms) 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Trunc (3001 ms) 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Remove (3009 ms) 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-09-13T13:31:37.877 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2975 ms) 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.RmXattr (2995 ms) 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3011 ms) 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] 14 tests from NeoRadosIo (40895 ms total) 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] Global test environment tear-down 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [==========] 14 tests from 1 test suite ran. (40895 ms total) 2024-09-13T13:31:37.878 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ PASSED ] 14 tests. 2024-09-13T13:31:37.882 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: Running main() from gmock_main.cc 2024-09-13T13:31:37.882 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-09-13T13:31:37.882 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] Global test environment set-up. 2024-09-13T13:31:37.882 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-09-13T13:31:37.882 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (2276 ms) 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3016 ms) 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3005 ms) 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (2518 ms) 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3037 ms) 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (2996 ms) 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3015 ms) 2024-09-13T13:31:37.883 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3026 ms) 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (2967 ms) 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2994 ms) 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3027 ms) 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2956 ms) 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3018 ms) 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-09-13T13:31:37.884 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (2989 ms) 2024-09-13T13:31:37.885 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (40840 ms total) 2024-09-13T13:31:37.885 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: 2024-09-13T13:31:37.885 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] Global test environment tear-down 2024-09-13T13:31:37.885 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (40840 ms total) 2024-09-13T13:31:37.885 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ PASSED ] 14 tests. 2024-09-13T13:31:39.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:39.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: osdmap e107: 8 total, 8 up, 8 in 2024-09-13T13:31:39.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:39.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:38 smithi189 ceph-mon[32912]: pgmap v109: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 11 op/s 2024-09-13T13:31:39.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: osdmap e107: 8 total, 8 up, 8 in 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:39.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[40255]: pgmap v109: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 11 op/s 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-76650-10"}]': finished 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: osdmap e107: 8 total, 8 up, 8 in 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:38 smithi043 ceph-mon[30106]: pgmap v109: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 11 op/s 2024-09-13T13:31:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: osdmap e108: 8 total, 8 up, 8 in 2024-09-13T13:31:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3637328751' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi043-76315-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:40.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2962179272' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:40.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: osdmap e108: 8 total, 8 up, 8 in 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3637328751' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi043-76315-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2962179272' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: osdmap e108: 8 total, 8 up, 8 in 2024-09-13T13:31:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3637328751' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi043-76315-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2962179272' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3637328751' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-76315-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: osdmap e109: 8 total, 8 up, 8 in 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: pgmap v112: 420 pgs: 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 53 KiB/s wr, 52 op/s 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-13T13:31:40.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:40.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-13T13:31:40.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[30106]: osdmap e110: 8 total, 8 up, 8 in 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3637328751' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-76315-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: osdmap e109: 8 total, 8 up, 8 in 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: pgmap v112: 420 pgs: 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 53 KiB/s wr, 52 op/s 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-13T13:31:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:40 smithi043 ceph-mon[40255]: osdmap e110: 8 total, 8 up, 8 in 2024-09-13T13:31:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3637328751' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-76315-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-76332-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: osdmap e109: 8 total, 8 up, 8 in 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: pgmap v112: 420 pgs: 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 53 KiB/s wr, 52 op/s 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-76650-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-76432-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-13T13:31:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:40 smithi189 ceph-mon[32912]: osdmap e110: 8 total, 8 up, 8 in 2024-09-13T13:31:41.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"}]': finished 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: osdmap e111: 8 total, 8 up, 8 in 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2767700061' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1603105075' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"}]': finished 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: osdmap e111: 8 total, 8 up, 8 in 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2767700061' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1603105075' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:41.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:41.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:41.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"} : dispatch 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pg_num","val":"11"}]': finished 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: osdmap e111: 8 total, 8 up, 8 in 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2767700061' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1603105075' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:42.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:42.293 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:42.293 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:42.293 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:42.293 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"} : dispatch 2024-09-13T13:31:43.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:42 smithi189 ceph-mon[32912]: pgmap v115: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 49 KiB/s wr, 49 op/s 2024-09-13T13:31:43.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:43.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:43.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"}]': finished 2024-09-13T13:31:43.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:42 smithi189 ceph-mon[32912]: osdmap e112: 8 total, 8 up, 8 in 2024-09-13T13:31:43.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[30106]: pgmap v115: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 49 KiB/s wr, 49 op/s 2024-09-13T13:31:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"}]': finished 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[30106]: osdmap e112: 8 total, 8 up, 8 in 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[40255]: pgmap v115: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 49 KiB/s wr, 49 op/s 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-76315-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-76332-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-76432-1","var":"pgp_num","val":"11"}]': finished 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[40255]: osdmap e112: 8 total, 8 up, 8 in 2024-09-13T13:31:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1412057760' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-13T13:31:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: osdmap e113: 8 total, 8 up, 8 in 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:44.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: osdmap e113: 8 total, 8 up, 8 in 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:31:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: osdmap e113: 8 total, 8 up, 8 in 2024-09-13T13:31:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:31:44.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:31:44.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:44.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:44.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:45.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:31:45.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: pgmap v118: 332 pgs: 40 unknown, 292 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:45.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"}]': finished 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: osdmap e114: 8 total, 8 up, 8 in 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/351231035' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"} : dispatch 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:45.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"} : dispatch 2024-09-13T13:31:45.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1221585331' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-76315-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: pgmap v118: 332 pgs: 40 unknown, 292 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"}]': finished 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: osdmap e114: 8 total, 8 up, 8 in 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/351231035' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"} : dispatch 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"} : dispatch 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1221585331' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-76315-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: pgmap v118: 332 pgs: 40 unknown, 292 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:31:45.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-13"}]': finished 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-76432-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: osdmap e114: 8 total, 8 up, 8 in 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/351231035' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"} : dispatch 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"} : dispatch 2024-09-13T13:31:45.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1221585331' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-76315-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:31:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:31:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:31:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:46 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"}]': finished 2024-09-13T13:31:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1221585331' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-76315-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:46 smithi189 ceph-mon[32912]: osdmap e115: 8 total, 8 up, 8 in 2024-09-13T13:31:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"}]': finished 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1221585331' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-76315-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[30106]: osdmap e115: 8 total, 8 up, 8 in 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-76332-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-13", "mode": "writeback"}]': finished 2024-09-13T13:31:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1221585331' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-76315-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:46.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:46 smithi043 ceph-mon[40255]: osdmap e115: 8 total, 8 up, 8 in 2024-09-13T13:31:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: pgmap v121: 396 pgs: 64 creating+peering, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 316 KiB/s wr, 44 op/s 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.0"} : dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.0"}]: dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.1"} : dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.1"}]: dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.2"} : dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.2"}]: dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.3"} : dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.3"}]: dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.4"} : dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.4"}]: dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.5"} : dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.5"}]: dispatch 2024-09-13T13:31:47.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.6"} : dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.6"}]: dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.7"} : dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.7"}]: dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.8"} : dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.8"}]: dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.9"} : dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.9"}]: dispatch 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: 173.9 scrub starts 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: 173.9 scrub ok 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:47.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:47 smithi189 ceph-mon[32912]: osdmap e116: 8 total, 8 up, 8 in 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: pgmap v121: 396 pgs: 64 creating+peering, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 316 KiB/s wr, 44 op/s 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.0"} : dispatch 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.0"}]: dispatch 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.1"} : dispatch 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.1"}]: dispatch 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.2"} : dispatch 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.2"}]: dispatch 2024-09-13T13:31:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.3"} : dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.3"}]: dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.4"} : dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.4"}]: dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.5"} : dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.5"}]: dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.6"} : dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.6"}]: dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.7"} : dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.7"}]: dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.8"} : dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.8"}]: dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.9"} : dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.9"}]: dispatch 2024-09-13T13:31:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: 173.9 scrub starts 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: 173.9 scrub ok 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[40255]: osdmap e116: 8 total, 8 up, 8 in 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: pgmap v121: 396 pgs: 64 creating+peering, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 316 KiB/s wr, 44 op/s 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.0"} : dispatch 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.0"}]: dispatch 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.1"} : dispatch 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.1"}]: dispatch 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.2"} : dispatch 2024-09-13T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.2"}]: dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.3"} : dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.3"}]: dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.4"} : dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.4"}]: dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.5"} : dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.5"}]: dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.6"} : dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.6"}]: dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.7"} : dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.7"}]: dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.8"} : dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.8"}]: dispatch 2024-09-13T13:31:47.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "173.9"} : dispatch 2024-09-13T13:31:47.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "173.9"}]: dispatch 2024-09-13T13:31:47.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: 173.9 scrub starts 2024-09-13T13:31:47.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: 173.9 scrub ok 2024-09-13T13:31:47.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:47.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-76432-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:47.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:47 smithi043 ceph-mon[30106]: osdmap e116: 8 total, 8 up, 8 in 2024-09-13T13:31:47.927 INFO:tasks.workunit.client.0.smithi043.stdout:d=3 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:c4fdafeb:::6:head expected=14:c4fdafeb:::6:head 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=6 expected=6 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:52ea6a34:::10:head expected=14:52ea6a34:::10:head 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=10 expected=10 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-09-13T13:31:47.928 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:89d3ae78:::11:head expected=14:89d3ae78:::11:head 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=11 expected=11 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:863748b0:::15:head 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:863748b0:::15:head expected=14:863748b0:::15:head 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=15 expected=15 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:6cac518f:::0:head expected=14:6cac518f:::0:head 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=0 expected=0 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-09-13T13:31:47.929 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:62a1935d:::14:head expected=14:62a1935d:::14:head 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=14 expected=14 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:566253c9:::13:head 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:566253c9:::13:head expected=14:566253c9:::13:head 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=13 expected=13 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=14:02547ec2:::1:head expected=14:02547ec2:::1:head 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=1 expected=1 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (221 ms) 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (35937 ms) 2024-09-13T13:31:47.930 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 0/5 -> MIN 14:33333333::::head 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 1/5 -> 14:33333333::::head 14:66666666::::head 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 2/5 -> 14:66666666::::head 14:99999999::::head 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 3/5 -> 14:99999999::::head 14:cccccccc::::head 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 4/5 -> 14:cccccccc::::head MAX 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (7845 ms) 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList (44629 ms total) 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1037 ms) 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-13T13:31:47.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-09-13T13:31:47.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-09-13T13:31:47.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-09-13T13:31:47.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-09-13T13:31:47.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-09-13T13:31:47.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-09-13T13:31:47.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (30 ms) 2024-09-13T13:31:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.8 scrub starts 2024-09-13T13:31:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.8 scrub ok 2024-09-13T13:31:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.0 scrub starts 2024-09-13T13:31:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.0 scrub ok 2024-09-13T13:31:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.6 scrub starts 2024-09-13T13:31:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.6 scrub ok 2024-09-13T13:31:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.2 scrub starts 2024-09-13T13:31:48.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: 173.2 scrub ok 2024-09-13T13:31:48.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: osdmap e117: 8 total, 8 up, 8 in 2024-09-13T13:31:48.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3311370593' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-76332-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1497955480' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.8 scrub starts 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.8 scrub ok 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.0 scrub starts 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.0 scrub ok 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.6 scrub starts 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.6 scrub ok 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.2 scrub starts 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: 173.2 scrub ok 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: osdmap e117: 8 total, 8 up, 8 in 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3311370593' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-76332-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1497955480' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.8 scrub starts 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.8 scrub ok 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.0 scrub starts 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.0 scrub ok 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.6 scrub starts 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.6 scrub ok 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.2 scrub starts 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: 173.2 scrub ok 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: osdmap e117: 8 total, 8 up, 8 in 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3311370593' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-76332-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1497955480' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: 173.1 scrub starts 2024-09-13T13:31:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: 173.1 scrub ok 2024-09-13T13:31:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: pgmap v124: 404 pgs: 104 unknown, 300 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-09-13T13:31:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: 173.4 scrub starts 2024-09-13T13:31:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: 173.4 scrub ok 2024-09-13T13:31:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3311370593' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-76332-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: osdmap e118: 8 total, 8 up, 8 in 2024-09-13T13:31:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: 173.1 scrub starts 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: 173.1 scrub ok 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: pgmap v124: 404 pgs: 104 unknown, 300 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: 173.4 scrub starts 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: 173.4 scrub ok 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3311370593' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-76332-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: osdmap e118: 8 total, 8 up, 8 in 2024-09-13T13:31:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: 173.1 scrub starts 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: 173.1 scrub ok 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: pgmap v124: 404 pgs: 104 unknown, 300 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: 173.4 scrub starts 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: 173.4 scrub ok 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3311370593' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-76332-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-76315-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: osdmap e118: 8 total, 8 up, 8 in 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:49.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: 173.3 scrub starts 2024-09-13T13:31:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: 173.3 scrub ok 2024-09-13T13:31:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: 173.5 scrub starts 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: 173.5 scrub ok 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: osdmap e119: 8 total, 8 up, 8 in 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:50.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: 173.3 scrub starts 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: 173.3 scrub ok 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: 173.5 scrub starts 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: 173.5 scrub ok 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: osdmap e119: 8 total, 8 up, 8 in 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:50.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: 173.3 scrub starts 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: 173.3 scrub ok 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: 173.5 scrub starts 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: 173.5 scrub ok 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: osdmap e119: 8 total, 8 up, 8 in 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/875139377' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:50.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:50.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"} : dispatch 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: pgmap v127: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: 173.7 scrub starts 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: 173.7 scrub ok 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: osdmap e120: 8 total, 8 up, 8 in 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/773718854' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2331890957' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.551 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: pgmap v127: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: 173.7 scrub starts 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: 173.7 scrub ok 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: osdmap e120: 8 total, 8 up, 8 in 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/773718854' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2331890957' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: pgmap v127: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: 173.7 scrub starts 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: 173.7 scrub ok 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-76432-2"}]': finished 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: osdmap e120: 8 total, 8 up, 8 in 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2353972801' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/773718854' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"} : dispatch 2024-09-13T13:31:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2331890957' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.List api_snapshots: Running main() from gmock_main.cc 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2013 ms) 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2004 ms) 2024-09-13T13:31:51.890 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2015 ms) 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2002 ms) 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8034 ms total) 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4081 ms) 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4112 ms) 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4886 ms) 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13079 ms total) 2024-09-13T13:31:51.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3003 ms) 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2004 ms) 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1992 ms) 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1996 ms) 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8995 ms total) 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-13T13:31:51.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3993 ms) 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4038 ms) 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8031 ms total) 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (55205 ms total) 2024-09-13T13:31:51.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-09-13T13:31:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:52 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:52 smithi189 ceph-mon[32912]: osdmap e121: 8 total, 8 up, 8 in 2024-09-13T13:31:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3964605493' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[30106]: osdmap e121: 8 total, 8 up, 8 in 2024-09-13T13:31:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3964605493' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-76650-15"}]': finished 2024-09-13T13:31:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-76332-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-76315-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[40255]: osdmap e121: 8 total, 8 up, 8 in 2024-09-13T13:31:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3964605493' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:52.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:53 smithi189 ceph-mon[32912]: pgmap v130: 388 pgs: 128 unknown, 260 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 73 op/s 2024-09-13T13:31:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:53 smithi189 ceph-mon[32912]: osdmap e122: 8 total, 8 up, 8 in 2024-09-13T13:31:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4107007352' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76709-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3964605493' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-13T13:31:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-13T13:31:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[30106]: pgmap v130: 388 pgs: 128 unknown, 260 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 73 op/s 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[30106]: osdmap e122: 8 total, 8 up, 8 in 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4107007352' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76709-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3964605493' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[40255]: pgmap v130: 388 pgs: 128 unknown, 260 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 73 op/s 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76432-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[40255]: osdmap e122: 8 total, 8 up, 8 in 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4107007352' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76709-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3964605493' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-13T13:31:53.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-13T13:31:53.901 INFO:tasks.workunit.client.0.smithi043.stdout:ObjectsStart 2024-09-13T13:31:53.901 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-09-13T13:31:53.901 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-09-13T13:31:53.902 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (43 ms) 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC (1110 ms total) 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3013 ms) 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP (3013 ms total) 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment tear-down 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] 11 tests from 3 test suites ran. (57267 ms total) 2024-09-13T13:31:53.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ PASSED ] 11 tests. 2024-09-13T13:31:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4107007352' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76709-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-13T13:31:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:54 smithi189 ceph-mon[32912]: osdmap e123: 8 total, 8 up, 8 in 2024-09-13T13:31:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/233275937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76315-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1999627013' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:54 smithi189 ceph-mon[32912]: pgmap v133: 388 pgs: 128 unknown, 260 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4107007352' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76709-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[30106]: osdmap e123: 8 total, 8 up, 8 in 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/233275937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76315-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1999627013' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[30106]: pgmap v133: 388 pgs: 128 unknown, 260 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4107007352' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76709-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-76432-3","pool2":"test-rados-api-smithi043-76432-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[40255]: osdmap e123: 8 total, 8 up, 8 in 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/233275937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76315-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1999627013' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:54 smithi043 ceph-mon[40255]: pgmap v133: 388 pgs: 128 unknown, 260 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:31:55.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:31:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:31:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:31:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:31:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/233275937' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76315-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1999627013' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[30106]: osdmap e124: 8 total, 8 up, 8 in 2024-09-13T13:31:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/233275937' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76315-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1999627013' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:55.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[40255]: osdmap e124: 8 total, 8 up, 8 in 2024-09-13T13:31:55.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:55 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:56.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/233275937' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76315-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:56.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1999627013' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:56.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:55 smithi189 ceph-mon[32912]: osdmap e124: 8 total, 8 up, 8 in 2024-09-13T13:31:56.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:55 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:31:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:56 smithi043 ceph-mon[30106]: osdmap e125: 8 total, 8 up, 8 in 2024-09-13T13:31:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:56 smithi043 ceph-mon[30106]: pgmap v136: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:31:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:56 smithi043 ceph-mon[30106]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:56 smithi043 ceph-mon[40255]: osdmap e125: 8 total, 8 up, 8 in 2024-09-13T13:31:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:56 smithi043 ceph-mon[40255]: pgmap v136: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:31:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:56 smithi043 ceph-mon[40255]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:56 smithi189 ceph-mon[32912]: osdmap e125: 8 total, 8 up, 8 in 2024-09-13T13:31:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:56 smithi189 ceph-mon[32912]: pgmap v136: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:31:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:56 smithi189 ceph-mon[32912]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[30106]: osdmap e126: 8 total, 8 up, 8 in 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/826373590' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2766455806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-76332-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[40255]: osdmap e126: 8 total, 8 up, 8 in 2024-09-13T13:31:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/826373590' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2766455806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-76332-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:57 smithi189 ceph-mon[32912]: osdmap e126: 8 total, 8 up, 8 in 2024-09-13T13:31:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/826373590' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2766455806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-76332-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:31:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2766455806' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-76332-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[30106]: osdmap e127: 8 total, 8 up, 8 in 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-76709-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[30106]: pgmap v139: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2766455806' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-76332-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[40255]: osdmap e127: 8 total, 8 up, 8 in 2024-09-13T13:31:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-76709-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:58 smithi043 ceph-mon[40255]: pgmap v139: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:31:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2766455806' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-76332-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-76315-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:31:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:31:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:58 smithi189 ceph-mon[32912]: osdmap e127: 8 total, 8 up, 8 in 2024-09-13T13:31:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-76709-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:31:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:58 smithi189 ceph-mon[32912]: pgmap v139: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:00.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[30106]: osdmap e128: 8 total, 8 up, 8 in 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-76709-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[30106]: osdmap e129: 8 total, 8 up, 8 in 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/179353885' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/456437729' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-76315-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[40255]: osdmap e128: 8 total, 8 up, 8 in 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-76709-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[40255]: osdmap e129: 8 total, 8 up, 8 in 2024-09-13T13:32:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/179353885' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:31:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/456437729' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-76315-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:59 smithi189 ceph-mon[32912]: osdmap e128: 8 total, 8 up, 8 in 2024-09-13T13:32:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-76709-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:59 smithi189 ceph-mon[32912]: osdmap e129: 8 total, 8 up, 8 in 2024-09-13T13:32:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/179353885' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:31:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/456437729' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-76315-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:01.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[30106]: pgmap v142: 364 pgs: 72 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/456437729' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-76315-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[30106]: osdmap e130: 8 total, 8 up, 8 in 2024-09-13T13:32:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[40255]: pgmap v142: 364 pgs: 72 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/456437729' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-76315-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:00 smithi043 ceph-mon[40255]: osdmap e130: 8 total, 8 up, 8 in 2024-09-13T13:32:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:00 smithi189 ceph-mon[32912]: pgmap v142: 364 pgs: 72 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-76332-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/456437729' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-76315-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:00 smithi189 ceph-mon[32912]: osdmap e130: 8 total, 8 up, 8 in 2024-09-13T13:32:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-13T13:32:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: osdmap e131: 8 total, 8 up, 8 in 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2"} : dispatch 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: osdmap e131: 8 total, 8 up, 8 in 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2"} : dispatch 2024-09-13T13:32:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: osdmap e131: 8 total, 8 up, 8 in 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2"} : dispatch 2024-09-13T13:32:02.335 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:02.336 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: pgmap v145: 300 pgs: 8 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: osdmap e132: 8 total, 8 up, 8 in 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3290636928' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi043-76315-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3295349573' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: pgmap v145: 300 pgs: 8 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: osdmap e132: 8 total, 8 up, 8 in 2024-09-13T13:32:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3290636928' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi043-76315-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3295349573' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: pgmap v145: 300 pgs: 8 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-76559-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: osdmap e132: 8 total, 8 up, 8 in 2024-09-13T13:32:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3290636928' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi043-76315-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3295349573' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-13T13:32:04.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: seed 76709 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2015 ms) 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (1998 ms) 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2018 ms) 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-13T13:32:04.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2000 ms) 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3538 ms) 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (11569 ms total) 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4188 ms) 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3878 ms) 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-13T13:32:04.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5916 ms) 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4038 ms) 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2030 ms) 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4917-gb0201b8c/rpm/el9/BUILD/ceph-19.3.0-4917-gb0201b8c/src/test/librados/snapshots_cxx.cc:460: Skipped 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi043-76709-7 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17931 ms) 2024-09-13T13:32:04.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37981 ms total) 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (10 ms) 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5048 ms) 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5058 ms total) 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2937 ms) 2024-09-13T13:32:04.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-13T13:32:05.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3290636928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-76315-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:05.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:05.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-13T13:32:05.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: osdmap e133: 8 total, 8 up, 8 in 2024-09-13T13:32:05.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:32:05.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:32:05.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: pgmap v148: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:32:05.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:04 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3290636928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-76315-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-13T13:32:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: osdmap e133: 8 total, 8 up, 8 in 2024-09-13T13:32:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:32:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:32:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: pgmap v148: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:32:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3290636928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-76315-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:05.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-76332-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:05.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-13T13:32:05.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: osdmap e133: 8 total, 8 up, 8 in 2024-09-13T13:32:05.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:32:05.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-13T13:32:05.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: pgmap v148: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:32:05.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:04 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:05.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:32:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:32:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:32:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:32:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-13T13:32:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[30106]: osdmap e134: 8 total, 8 up, 8 in 2024-09-13T13:32:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:32:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:32:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-13T13:32:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[40255]: osdmap e134: 8 total, 8 up, 8 in 2024-09-13T13:32:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:32:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:32:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:05 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-13T13:32:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:05 smithi189 ceph-mon[32912]: osdmap e134: 8 total, 8 up, 8 in 2024-09-13T13:32:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2947825924' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:32:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"} : dispatch 2024-09-13T13:32:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:05 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRados api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-13T13:32:06.891 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: seed 76559 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (397 ms) 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (15 ms) 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (8 ms) 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (10 ms) 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (3 ms) 2024-09-13T13:32:06.892 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (6 ms) 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (13 ms) 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (10 ms) 2024-09-13T13:32:06.893 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1834 ms) 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (401 ms) 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting, doing copies 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60606 ms) 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-13T13:32:06.894 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (4 ms) 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4250 ms) 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-13T13:32:06.895 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-13T13:32:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"}]': finished 2024-09-13T13:32:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:06 smithi189 ceph-mon[32912]: osdmap e135: 8 total, 8 up, 8 in 2024-09-13T13:32:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3487378287' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-76315-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1083075436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-76332-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:06 smithi189 ceph-mon[32912]: pgmap v151: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:32:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:06 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"}]': finished 2024-09-13T13:32:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[30106]: osdmap e135: 8 total, 8 up, 8 in 2024-09-13T13:32:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3487378287' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-76315-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1083075436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-76332-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[30106]: pgmap v151: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:32:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-76559-1","app":"app1","key":"key1"}]': finished 2024-09-13T13:32:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[40255]: osdmap e135: 8 total, 8 up, 8 in 2024-09-13T13:32:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3487378287' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-76315-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1083075436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-76332-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[40255]: pgmap v151: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:32:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:06 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3487378287' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-76315-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1083075436' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-76332-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:07 smithi189 ceph-mon[32912]: osdmap e136: 8 total, 8 up, 8 in 2024-09-13T13:32:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76559-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:08.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3487378287' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-76315-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1083075436' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-76332-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[40255]: osdmap e136: 8 total, 8 up, 8 in 2024-09-13T13:32:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76559-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3487378287' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-76315-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1083075436' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-76332-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:08.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[30106]: osdmap e136: 8 total, 8 up, 8 in 2024-09-13T13:32:08.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:08.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:08.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76559-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:09.148 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:32:08 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=infra.usagestats t=2024-09-13T13:32:08.681331313Z level=info msg="Usage stats are ready to report" 2024-09-13T13:32:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76559-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:08 smithi189 ceph-mon[32912]: osdmap e137: 8 total, 8 up, 8 in 2024-09-13T13:32:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76559-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:08 smithi189 ceph-mon[32912]: pgmap v154: 268 pgs: 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:32:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76559-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[30106]: osdmap e137: 8 total, 8 up, 8 in 2024-09-13T13:32:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76559-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[30106]: pgmap v154: 268 pgs: 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:32:09.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76559-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:09.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[40255]: osdmap e137: 8 total, 8 up, 8 in 2024-09-13T13:32:09.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76559-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:09.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:08 smithi043 ceph-mon[40255]: pgmap v154: 268 pgs: 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-13T13:32:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:09 smithi189 ceph-mon[32912]: osdmap e138: 8 total, 8 up, 8 in 2024-09-13T13:32:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2103551518' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3654697564' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-76315-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[30106]: osdmap e138: 8 total, 8 up, 8 in 2024-09-13T13:32:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2103551518' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3654697564' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-76315-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[40255]: osdmap e138: 8 total, 8 up, 8 in 2024-09-13T13:32:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2103551518' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3654697564' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-76315-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:10.921 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP snapshots: Running main() from gmock_main.cc 2024-09-13T13:32:10.921 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-09-13T13:32:10.921 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] Global test environment set-up. 2024-09-13T13:32:10.921 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-09-13T13:32:10.921 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (4318 ms) 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5001 ms) 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (3519 ms) 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5048 ms) 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (6944 ms) 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (24830 ms total) 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: 2024-09-13T13:32:10.922 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-09-13T13:32:10.923 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-09-13T13:32:10.923 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5005 ms) 2024-09-13T13:32:10.923 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-09-13T13:32:10.923 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (5993 ms) 2024-09-13T13:32:10.923 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-09-13T13:32:10.923 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8012 ms) 2024-09-13T13:32:10.924 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-09-13T13:32:10.924 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5987 ms) 2024-09-13T13:32:10.924 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-09-13T13:32:10.924 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4019 ms) 2024-09-13T13:32:10.924 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-09-13T13:32:10.924 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi043-77698-11. 2024-09-13T13:32:10.925 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: Waiting for snaps to purge. 2024-09-13T13:32:10.925 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20032 ms) 2024-09-13T13:32:10.925 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49049 ms total) 2024-09-13T13:32:10.925 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: 2024-09-13T13:32:10.925 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] Global test environment tear-down 2024-09-13T13:32:10.925 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (73879 ms total) 2024-09-13T13:32:10.925 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ PASSED ] 11 tests. 2024-09-13T13:32:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76559-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3654697564' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-76315-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[30106]: osdmap e139: 8 total, 8 up, 8 in 2024-09-13T13:32:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:32:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[30106]: pgmap v157: 332 pgs: 8 creating+peering, 324 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76559-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3654697564' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-76315-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[40255]: osdmap e139: 8 total, 8 up, 8 in 2024-09-13T13:32:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:32:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:10 smithi043 ceph-mon[40255]: pgmap v157: 332 pgs: 8 creating+peering, 324 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76559-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-76332-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3654697564' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-76315-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:10 smithi189 ceph-mon[32912]: osdmap e139: 8 total, 8 up, 8 in 2024-09-13T13:32:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:32:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:10 smithi189 ceph-mon[32912]: pgmap v157: 332 pgs: 8 creating+peering, 324 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[30106]: osdmap e140: 8 total, 8 up, 8 in 2024-09-13T13:32:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:32:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[40255]: osdmap e140: 8 total, 8 up, 8 in 2024-09-13T13:32:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:32:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:11 smithi189 ceph-mon[32912]: osdmap e140: 8 total, 8 up, 8 in 2024-09-13T13:32:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"} : dispatch 2024-09-13T13:32:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: osdmap e141: 8 total, 8 up, 8 in 2024-09-13T13:32:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1767948763' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412959317' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: pgmap v160: 332 pgs: 96 unknown, 8 creating+peering, 228 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: osdmap e141: 8 total, 8 up, 8 in 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1767948763' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412959317' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: pgmap v160: 332 pgs: 96 unknown, 8 creating+peering, 228 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: osdmap e142: 8 total, 8 up, 8 in 2024-09-13T13:32:13.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:13.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: osdmap e142: 8 total, 8 up, 8 in 2024-09-13T13:32:13.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/576181846' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-76709-16"}]': finished 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: osdmap e141: 8 total, 8 up, 8 in 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1767948763' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412959317' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: pgmap v160: 332 pgs: 96 unknown, 8 creating+peering, 228 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-76315-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-76332-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76559-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: osdmap e142: 8 total, 8 up, 8 in 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:13.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:13 smithi043 ceph-mon[30106]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:13 smithi043 ceph-mon[30106]: osdmap e143: 8 total, 8 up, 8 in 2024-09-13T13:32:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:13 smithi043 ceph-mon[40255]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:13 smithi043 ceph-mon[40255]: osdmap e143: 8 total, 8 up, 8 in 2024-09-13T13:32:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:13 smithi189 ceph-mon[32912]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:13 smithi189 ceph-mon[32912]: osdmap e143: 8 total, 8 up, 8 in 2024-09-13T13:32:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-76315-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: pgmap v163: 236 pgs: 8 creating+peering, 228 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-76315-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: osdmap e144: 8 total, 8 up, 8 in 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-76315-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1023896526' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-76332-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-76315-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: pgmap v163: 236 pgs: 8 creating+peering, 228 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-76315-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: osdmap e144: 8 total, 8 up, 8 in 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-76315-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:15.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1023896526' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-76332-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-76315-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: pgmap v163: 236 pgs: 8 creating+peering, 228 active+clean; 590 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-76315-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: osdmap e144: 8 total, 8 up, 8 in 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-76315-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:15.317 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1023896526' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-76332-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:15.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:32:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:32:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:32:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:32:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1023896526' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-76332-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[40255]: osdmap e145: 8 total, 8 up, 8 in 2024-09-13T13:32:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1023896526' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-76332-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[30106]: osdmap e145: 8 total, 8 up, 8 in 2024-09-13T13:32:15.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:15 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1023896526' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-76332-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:15 smithi189 ceph-mon[32912]: osdmap e145: 8 total, 8 up, 8 in 2024-09-13T13:32:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"} : dispatch 2024-09-13T13:32:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[30106]: pgmap v166: 268 pgs: 40 creating+peering, 228 active+clean; 590 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-76315-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[30106]: osdmap e146: 8 total, 8 up, 8 in 2024-09-13T13:32:17.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[40255]: pgmap v166: 268 pgs: 40 creating+peering, 228 active+clean; 590 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:17.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-76315-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:17.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:17.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:16 smithi043 ceph-mon[40255]: osdmap e146: 8 total, 8 up, 8 in 2024-09-13T13:32:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:16 smithi189 ceph-mon[32912]: pgmap v166: 268 pgs: 40 creating+peering, 228 active+clean; 590 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-76315-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/194089457' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76559-24"}]': finished 2024-09-13T13:32:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:16 smithi189 ceph-mon[32912]: osdmap e146: 8 total, 8 up, 8 in 2024-09-13T13:32:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[40255]: osdmap e147: 8 total, 8 up, 8 in 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/304217922' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4291014247' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[40255]: pgmap v169: 308 pgs: 72 unknown, 8 creating+peering, 228 active+clean; 590 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[30106]: osdmap e147: 8 total, 8 up, 8 in 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/304217922' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4291014247' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:18 smithi043 ceph-mon[30106]: pgmap v169: 308 pgs: 72 unknown, 8 creating+peering, 228 active+clean; 590 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:18 smithi189 ceph-mon[32912]: osdmap e147: 8 total, 8 up, 8 in 2024-09-13T13:32:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/304217922' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4291014247' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:18 smithi189 ceph-mon[32912]: pgmap v169: 308 pgs: 72 unknown, 8 creating+peering, 228 active+clean; 590 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[40255]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[40255]: osdmap e148: 8 total, 8 up, 8 in 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[30106]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[30106]: osdmap e148: 8 total, 8 up, 8 in 2024-09-13T13:32:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:20.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:19 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:19 smithi189 ceph-mon[32912]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-76559-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-76332-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:19 smithi189 ceph-mon[32912]: osdmap e148: 8 total, 8 up, 8 in 2024-09-13T13:32:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:19 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: Running main() from gmock_main.cc 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment set-up. 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.TooBig (2760 ms) 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3020 ms) 2024-09-13T13:32:20.950 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3007 ms) 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2496 ms) 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3047 ms) 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3010 ms) 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3012 ms) 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3042 ms) 2024-09-13T13:32:20.951 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2932 ms) 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2989 ms) 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3065 ms) 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2926 ms) 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3030 ms) 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2975 ms) 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.Flush (3010 ms) 2024-09-13T13:32:20.952 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2977 ms) 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3008 ms) 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3010 ms) 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3022 ms) 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3007 ms) 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3020 ms) 2024-09-13T13:32:20.953 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (2933 ms) 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3010 ms) 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3021 ms) 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3036 ms) 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3008 ms) 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 26 tests from LibRadosAio (77374 ms total) 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-13T13:32:20.954 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-13T13:32:21.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:21.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[30106]: osdmap e149: 8 total, 8 up, 8 in 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[30106]: pgmap v172: 236 pgs: 236 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[40255]: osdmap e149: 8 total, 8 up, 8 in 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[40255]: pgmap v172: 236 pgs: 236 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-13T13:32:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:21.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:20 smithi189 ceph-mon[32912]: osdmap e149: 8 total, 8 up, 8 in 2024-09-13T13:32:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"} : dispatch 2024-09-13T13:32:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:20 smithi189 ceph-mon[32912]: pgmap v172: 236 pgs: 236 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-13T13:32:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3901000595' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-76315-27"}]': finished 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: osdmap e150: 8 total, 8 up, 8 in 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2341776710' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-76559-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3932007544' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"}]': finished 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2341776710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-76559-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: osdmap e151: 8 total, 8 up, 8 in 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: osdmap e150: 8 total, 8 up, 8 in 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2341776710' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-76559-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3932007544' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"}]': finished 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2341776710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-76559-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: osdmap e151: 8 total, 8 up, 8 in 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:32:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: osdmap e150: 8 total, 8 up, 8 in 2024-09-13T13:32:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2341776710' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-76559-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3932007544' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-13"}]': finished 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2341776710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-76559-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-76332-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:22.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-76315-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:22.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: osdmap e151: 8 total, 8 up, 8 in 2024-09-13T13:32:22.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:22.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:22 smithi043 ceph-mon[30106]: pgmap v175: 300 pgs: 64 unknown, 236 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-13T13:32:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:22 smithi043 ceph-mon[30106]: osdmap e152: 8 total, 8 up, 8 in 2024-09-13T13:32:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:22 smithi043 ceph-mon[40255]: pgmap v175: 300 pgs: 64 unknown, 236 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-13T13:32:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:22 smithi043 ceph-mon[40255]: osdmap e152: 8 total, 8 up, 8 in 2024-09-13T13:32:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:22 smithi189 ceph-mon[32912]: pgmap v175: 300 pgs: 64 unknown, 236 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-09-13T13:32:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:22 smithi189 ceph-mon[32912]: osdmap e152: 8 total, 8 up, 8 in 2024-09-13T13:32:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: osdmap e153: 8 total, 8 up, 8 in 2024-09-13T13:32:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1397445864' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-76332-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3115908780' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:24 smithi189 ceph-mon[32912]: pgmap v178: 308 pgs: 104 unknown, 204 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:25.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: osdmap e153: 8 total, 8 up, 8 in 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1397445864' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-76332-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3115908780' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[40255]: pgmap v178: 308 pgs: 104 unknown, 204 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-76315-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: osdmap e153: 8 total, 8 up, 8 in 2024-09-13T13:32:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1397445864' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-76332-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3115908780' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:24 smithi043 ceph-mon[30106]: pgmap v178: 308 pgs: 104 unknown, 204 active+clean; 608 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:25.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:32:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:32:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:32:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:32:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[30106]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1397445864' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-76332-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[30106]: osdmap e154: 8 total, 8 up, 8 in 2024-09-13T13:32:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[40255]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:25.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:25.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1397445864' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-76332-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:25.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:25.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[40255]: osdmap e154: 8 total, 8 up, 8 in 2024-09-13T13:32:25.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:25 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:25 smithi189 ceph-mon[32912]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1397445864' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-76332-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-76559-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:25 smithi189 ceph-mon[32912]: osdmap e154: 8 total, 8 up, 8 in 2024-09-13T13:32:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:25 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: osdmap e155: 8 total, 8 up, 8 in 2024-09-13T13:32:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:26 smithi189 ceph-mon[32912]: pgmap v181: 236 pgs: 236 active+clean; 616 KiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:27.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: osdmap e155: 8 total, 8 up, 8 in 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[30106]: pgmap v181: 236 pgs: 236 active+clean; 616 KiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: osdmap e155: 8 total, 8 up, 8 in 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:27.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:26 smithi043 ceph-mon[40255]: pgmap v181: 236 pgs: 236 active+clean; 616 KiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: osdmap e156: 8 total, 8 up, 8 in 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2425375216' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-76332-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: osdmap e156: 8 total, 8 up, 8 in 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2425375216' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-76332-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: osdmap e156: 8 total, 8 up, 8 in 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3677264890' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2425375216' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-76332-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"} : dispatch 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:28.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2425375216' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-76332-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"}]': finished 2024-09-13T13:32:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"} : dispatch 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: osdmap e157: 8 total, 8 up, 8 in 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"} : dispatch 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-76315-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:29.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:28 smithi189 ceph-mon[32912]: pgmap v184: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2425375216' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-76332-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"}]': finished 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"} : dispatch 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: osdmap e157: 8 total, 8 up, 8 in 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"} : dispatch 2024-09-13T13:32:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-76315-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[30106]: pgmap v184: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2425375216' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-76332-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-76315-28"}]': finished 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-15"}]': finished 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"} : dispatch 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: osdmap e157: 8 total, 8 up, 8 in 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"} : dispatch 2024-09-13T13:32:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:29.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:29.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:29.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-76315-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:29.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:28 smithi043 ceph-mon[40255]: pgmap v184: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-13T13:32:29.913 INFO:tasks.workunit.client.0.smithi043.stdout: SnapshotsECPP.SnapRemovePP (2007 ms) 2024-09-13T13:32:29.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-13T13:32:29.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2017 ms) 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2023 ms) 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8984 ms total) 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4049 ms) 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3900 ms) 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4013 ms) 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11962 ms total) 2024-09-13T13:32:29.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-09-13T13:32:29.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-13T13:32:29.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (93183 ms total) 2024-09-13T13:32:29.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-13T13:32:29.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-09-13T13:32:29.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-13T13:32:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"}]': finished 2024-09-13T13:32:30.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:30.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-76315-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:30.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: osdmap e158: 8 total, 8 up, 8 in 2024-09-13T13:32:30.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi043-76315-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:30.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:30.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"}]': finished 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-76315-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: osdmap e158: 8 total, 8 up, 8 in 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi043-76315-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-76559-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-15", "mode": "writeback"}]': finished 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-76315-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: osdmap e158: 8 total, 8 up, 8 in 2024-09-13T13:32:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi043-76315-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:30.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/710189251' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:30.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"} : dispatch 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[30106]: osdmap e159: 8 total, 8 up, 8 in 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1118046030' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-76332-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[30106]: pgmap v187: 268 pgs: 32 creating+peering, 236 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[40255]: osdmap e159: 8 total, 8 up, 8 in 2024-09-13T13:32:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1118046030' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-76332-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:30 smithi043 ceph-mon[40255]: pgmap v187: 268 pgs: 32 creating+peering, 236 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-76709-21"}]': finished 2024-09-13T13:32:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:30 smithi189 ceph-mon[32912]: osdmap e159: 8 total, 8 up, 8 in 2024-09-13T13:32:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1118046030' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-76332-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:31.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:30 smithi189 ceph-mon[32912]: pgmap v187: 268 pgs: 32 creating+peering, 236 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:31.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (2744 ms) 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (14580 ms) 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6044 ms) 2024-09-13T13:32:31.867 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2944 ms) 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3004 ms) 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3005 ms) 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2979 ms) 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2996 ms) 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-13T13:32:31.868 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3009 ms) 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3017 ms) 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2972 ms) 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3010 ms) 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3008 ms) 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3015 ms) 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-13T13:32:31.869 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3014 ms) 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3020 ms) 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2939 ms) 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3010 ms) 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3015 ms) 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3026 ms) 2024-09-13T13:32:31.870 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3016 ms) 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3004 ms) 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3004 ms) 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2937 ms) 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3004 ms) 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3022 ms) 2024-09-13T13:32:31.871 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-13T13:32:32.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-76315-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1118046030' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-76332-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: osdmap e160: 8 total, 8 up, 8 in 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"}]': finished 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: osdmap e161: 8 total, 8 up, 8 in 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-76315-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1118046030' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-76332-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: osdmap e160: 8 total, 8 up, 8 in 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"}]': finished 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: osdmap e161: 8 total, 8 up, 8 in 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-76315-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1118046030' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-76332-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: osdmap e160: 8 total, 8 up, 8 in 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"} : dispatch 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-15"}]': finished 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: osdmap e161: 8 total, 8 up, 8 in 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/14496067' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"} : dispatch 2024-09-13T13:32:32.876 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio (67573 ms total) 2024-09-13T13:32:32.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-13T13:32:32.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-13T13:32:32.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-13T13:32:32.877 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (108 ms) 2024-09-13T13:32:32.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (108 ms total) 2024-09-13T13:32:32.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-13T13:32:32.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-13T13:32:32.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-13T13:32:32.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (43 ms) 2024-09-13T13:32:32.878 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-13T13:32:32.879 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (7 ms) 2024-09-13T13:32:32.879 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (51 ms total) 2024-09-13T13:32:32.879 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-13T13:32:32.879 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-13T13:32:32.879 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-13T13:32:32.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (37 ms) 2024-09-13T13:32:32.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-13T13:32:32.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (3 ms) 2024-09-13T13:32:32.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (40 ms total) 2024-09-13T13:32:32.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-13T13:32:32.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-13T13:32:32.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-13T13:32:32.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (44 ms) 2024-09-13T13:32:32.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-13T13:32:32.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (8 ms) 2024-09-13T13:32:32.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (52 ms total) 2024-09-13T13:32:32.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-13T13:32:32.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-13T13:32:32.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-13T13:32:32.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1021 ms) 2024-09-13T13:32:32.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1021 ms total) 2024-09-13T13:32:32.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-09-13T13:32:32.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-13T13:32:32.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (96203 ms total) 2024-09-13T13:32:32.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-13T13:32:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[30106]: pgmap v190: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[30106]: osdmap e162: 8 total, 8 up, 8 in 2024-09-13T13:32:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1486910329' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[40255]: pgmap v190: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:33.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[40255]: osdmap e162: 8 total, 8 up, 8 in 2024-09-13T13:32:33.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1486910329' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:33.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:33.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:32 smithi189 ceph-mon[32912]: pgmap v190: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-76559-36"}]': finished 2024-09-13T13:32:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:32 smithi189 ceph-mon[32912]: osdmap e162: 8 total, 8 up, 8 in 2024-09-13T13:32:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1486910329' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:35.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: osdmap e163: 8 total, 8 up, 8 in 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: pgmap v193: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:32:35.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:35.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: osdmap e163: 8 total, 8 up, 8 in 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: pgmap v193: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-76332-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: osdmap e163: 8 total, 8 up, 8 in 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"} : dispatch 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: pgmap v193: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:32:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:35.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:32:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:32:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: osdmap e164: 8 total, 8 up, 8 in 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:35.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:35.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:35.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:35.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:35.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: osdmap e164: 8 total, 8 up, 8 in 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:35.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:35 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3435424996' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-76315-29"}]': finished 2024-09-13T13:32:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: osdmap e164: 8 total, 8 up, 8 in 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:35 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: osdmap e165: 8 total, 8 up, 8 in 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1269337351' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-76332-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: pgmap v196: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"}]': finished 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: osdmap e165: 8 total, 8 up, 8 in 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1269337351' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-76332-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:37.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: pgmap v196: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:37.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:37.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"}]': finished 2024-09-13T13:32:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-76315-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: osdmap e165: 8 total, 8 up, 8 in 2024-09-13T13:32:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1269337351' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-76332-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: pgmap v196: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-17"}]': finished 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1269337351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-76332-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"} : dispatch 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: osdmap e166: 8 total, 8 up, 8 in 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"} : dispatch 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"}]': finished 2024-09-13T13:32:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[30106]: osdmap e167: 8 total, 8 up, 8 in 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1269337351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-76332-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"} : dispatch 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: osdmap e166: 8 total, 8 up, 8 in 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"} : dispatch 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"}]': finished 2024-09-13T13:32:38.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:37 smithi043 ceph-mon[40255]: osdmap e167: 8 total, 8 up, 8 in 2024-09-13T13:32:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1269337351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-76332-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"} : dispatch 2024-09-13T13:32:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: osdmap e166: 8 total, 8 up, 8 in 2024-09-13T13:32:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"} : dispatch 2024-09-13T13:32:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-76315-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-17", "mode": "writeback"}]': finished 2024-09-13T13:32:38.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:37 smithi189 ceph-mon[32912]: osdmap e167: 8 total, 8 up, 8 in 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: pgmap v199: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: osdmap e168: 8 total, 8 up, 8 in 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3189840036' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: pgmap v199: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: osdmap e168: 8 total, 8 up, 8 in 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3189840036' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: pgmap v199: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: osdmap e168: 8 total, 8 up, 8 in 2024-09-13T13:32:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"} : dispatch 2024-09-13T13:32:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3189840036' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"}]': finished 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[30106]: osdmap e169: 8 total, 8 up, 8 in 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"}]': finished 2024-09-13T13:32:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[40255]: osdmap e169: 8 total, 8 up, 8 in 2024-09-13T13:32:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:39 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-17"}]': finished 2024-09-13T13:32:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-76332-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:39 smithi189 ceph-mon[32912]: osdmap e169: 8 total, 8 up, 8 in 2024-09-13T13:32:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[30106]: pgmap v202: 260 pgs: 260 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[30106]: osdmap e170: 8 total, 8 up, 8 in 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[40255]: pgmap v202: 260 pgs: 260 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[40255]: osdmap e170: 8 total, 8 up, 8 in 2024-09-13T13:32:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:41.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:40 smithi189 ceph-mon[32912]: pgmap v202: 260 pgs: 260 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-13T13:32:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:40 smithi189 ceph-mon[32912]: osdmap e170: 8 total, 8 up, 8 in 2024-09-13T13:32:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1720065762' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"} : dispatch 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[30106]: osdmap e171: 8 total, 8 up, 8 in 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1753292422' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:42.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:42.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[40255]: osdmap e171: 8 total, 8 up, 8 in 2024-09-13T13:32:42.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1753292422' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:41 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-76315-30"}]': finished 2024-09-13T13:32:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:41 smithi189 ceph-mon[32912]: osdmap e171: 8 total, 8 up, 8 in 2024-09-13T13:32:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1753292422' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-76315-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: pgmap v205: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-76315-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: osdmap e172: 8 total, 8 up, 8 in 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-76315-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-76315-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: pgmap v205: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-76315-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: osdmap e172: 8 total, 8 up, 8 in 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-76315-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-76315-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: pgmap v205: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-76332-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-76315-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:43.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: osdmap e172: 8 total, 8 up, 8 in 2024-09-13T13:32:43.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-76315-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:43.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:43.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1753292422' entity='client.admin' cmd={ 2024-09-13T13:32:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "prefix": "osd pool set", 2024-09-13T13:32:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "var": "eio", 2024-09-13T13:32:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "val": "true" 2024-09-13T13:32:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: } : dispatch 2024-09-13T13:32:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={ 2024-09-13T13:32:44.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "prefix": "osd pool set", 2024-09-13T13:32:44.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "var": "eio", 2024-09-13T13:32:44.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "val": "true" 2024-09-13T13:32:44.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: } : dispatch 2024-09-13T13:32:44.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:44.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{ 2024-09-13T13:32:44.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "prefix": "osd pool set", 2024-09-13T13:32:44.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "var": "eio", 2024-09-13T13:32:44.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: "val": "true" 2024-09-13T13:32:44.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: }]': finished 2024-09-13T13:32:44.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:44.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: osdmap e173: 8 total, 8 up, 8 in 2024-09-13T13:32:44.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:44.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1753292422' entity='client.admin' cmd={ 2024-09-13T13:32:44.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "prefix": "osd pool set", 2024-09-13T13:32:44.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "var": "eio", 2024-09-13T13:32:44.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "val": "true" 2024-09-13T13:32:44.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: } : dispatch 2024-09-13T13:32:44.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={ 2024-09-13T13:32:44.248 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "prefix": "osd pool set", 2024-09-13T13:32:44.248 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.248 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "var": "eio", 2024-09-13T13:32:44.248 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "val": "true" 2024-09-13T13:32:44.248 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: } : dispatch 2024-09-13T13:32:44.249 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:44.249 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{ 2024-09-13T13:32:44.249 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "prefix": "osd pool set", 2024-09-13T13:32:44.249 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.249 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "var": "eio", 2024-09-13T13:32:44.249 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: "val": "true" 2024-09-13T13:32:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: }]': finished 2024-09-13T13:32:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: osdmap e173: 8 total, 8 up, 8 in 2024-09-13T13:32:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1753292422' entity='client.admin' cmd={ 2024-09-13T13:32:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "prefix": "osd pool set", 2024-09-13T13:32:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "var": "eio", 2024-09-13T13:32:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "val": "true" 2024-09-13T13:32:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: } : dispatch 2024-09-13T13:32:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={ 2024-09-13T13:32:44.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "prefix": "osd pool set", 2024-09-13T13:32:44.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "var": "eio", 2024-09-13T13:32:44.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "val": "true" 2024-09-13T13:32:44.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: } : dispatch 2024-09-13T13:32:44.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:44.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{ 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "prefix": "osd pool set", 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "pool": "PoolEIOFlag_smithi043-76332-33", 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "var": "eio", 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: "val": "true" 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: }]': finished 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: osdmap e173: 8 total, 8 up, 8 in 2024-09-13T13:32:44.405 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:45.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[40255]: pgmap v208: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:45.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-76315-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"}]': finished 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[40255]: osdmap e174: 8 total, 8 up, 8 in 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"} : dispatch 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"} : dispatch 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[30106]: pgmap v208: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-76315-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"}]': finished 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[30106]: osdmap e174: 8 total, 8 up, 8 in 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"} : dispatch 2024-09-13T13:32:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"} : dispatch 2024-09-13T13:32:45.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:44 smithi189 ceph-mon[32912]: pgmap v208: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:45.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-76315-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:45.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-19"}]': finished 2024-09-13T13:32:45.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:44 smithi189 ceph-mon[32912]: osdmap e174: 8 total, 8 up, 8 in 2024-09-13T13:32:45.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"} : dispatch 2024-09-13T13:32:45.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"} : dispatch 2024-09-13T13:32:45.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:32:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:32:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"}]': finished 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: osdmap e175: 8 total, 8 up, 8 in 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3146983613' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi043-76332-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"}]': finished 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: osdmap e175: 8 total, 8 up, 8 in 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3146983613' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi043-76332-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:32:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:32:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:32:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:32:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-19", "mode": "writeback"}]': finished 2024-09-13T13:32:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: osdmap e175: 8 total, 8 up, 8 in 2024-09-13T13:32:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3146983613' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi043-76332-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[30106]: pgmap v211: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3146983613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-76332-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[30106]: osdmap e176: 8 total, 8 up, 8 in 2024-09-13T13:32:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:47.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[40255]: pgmap v211: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:47.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3146983613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-76332-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:47.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:47.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[40255]: osdmap e176: 8 total, 8 up, 8 in 2024-09-13T13:32:47.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:47.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:47.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:46 smithi189 ceph-mon[32912]: pgmap v211: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3146983613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-76332-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:46 smithi189 ceph-mon[32912]: osdmap e176: 8 total, 8 up, 8 in 2024-09-13T13:32:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"} : dispatch 2024-09-13T13:32:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:48.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"}]': finished 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[30106]: osdmap e177: 8 total, 8 up, 8 in 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"}]': finished 2024-09-13T13:32:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:48.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[40255]: osdmap e177: 8 total, 8 up, 8 in 2024-09-13T13:32:48.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:47 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:47 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-19"}]': finished 2024-09-13T13:32:48.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:48.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:47 smithi189 ceph-mon[32912]: osdmap e177: 8 total, 8 up, 8 in 2024-09-13T13:32:48.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: pgmap v214: 228 pgs: 228 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: osdmap e178: 8 total, 8 up, 8 in 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3074502715' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: pgmap v214: 228 pgs: 228 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: osdmap e178: 8 total, 8 up, 8 in 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3074502715' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:49.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: pgmap v214: 228 pgs: 228 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:32:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2993461034' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-76315-31"}]': finished 2024-09-13T13:32:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: osdmap e178: 8 total, 8 up, 8 in 2024-09-13T13:32:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3074502715' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:50.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: osdmap e179: 8 total, 8 up, 8 in 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:50.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: osdmap e179: 8 total, 8 up, 8 in 2024-09-13T13:32:50.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:50.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:50.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:50.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:32:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-76332-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-76315-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: osdmap e179: 8 total, 8 up, 8 in 2024-09-13T13:32:50.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:50.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:50.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:50.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[40255]: pgmap v217: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:32:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[40255]: osdmap e180: 8 total, 8 up, 8 in 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[30106]: pgmap v217: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[30106]: osdmap e180: 8 total, 8 up, 8 in 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:51.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:50 smithi189 ceph-mon[32912]: pgmap v217: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:32:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:50 smithi189 ceph-mon[32912]: osdmap e180: 8 total, 8 up, 8 in 2024-09-13T13:32:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:32:53.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: osdmap e181: 8 total, 8 up, 8 in 2024-09-13T13:32:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3010742623' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[40255]: pgmap v220: 268 pgs: 40 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: osdmap e181: 8 total, 8 up, 8 in 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3010742623' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:53.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:52 smithi043 ceph-mon[30106]: pgmap v220: 268 pgs: 40 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:53.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-76315-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:32:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: osdmap e181: 8 total, 8 up, 8 in 2024-09-13T13:32:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3010742623' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:52 smithi189 ceph-mon[32912]: pgmap v220: 268 pgs: 40 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:32:54.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"}]': finished 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[40255]: osdmap e182: 8 total, 8 up, 8 in 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"} : dispatch 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"} : dispatch 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"}]': finished 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[30106]: osdmap e182: 8 total, 8 up, 8 in 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"} : dispatch 2024-09-13T13:32:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"} : dispatch 2024-09-13T13:32:54.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:53 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:32:54.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-21"}]': finished 2024-09-13T13:32:54.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:54.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:53 smithi189 ceph-mon[32912]: osdmap e182: 8 total, 8 up, 8 in 2024-09-13T13:32:54.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"} : dispatch 2024-09-13T13:32:54.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"} : dispatch 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"}]': finished 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: osdmap e183: 8 total, 8 up, 8 in 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3190262686' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[40255]: pgmap v223: 292 pgs: 64 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"}]': finished 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: osdmap e183: 8 total, 8 up, 8 in 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3190262686' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:55.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:55.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:54 smithi043 ceph-mon[30106]: pgmap v223: 292 pgs: 64 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:55.361 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:32:55.361 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-21", "mode": "writeback"}]': finished 2024-09-13T13:32:55.361 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: osdmap e183: 8 total, 8 up, 8 in 2024-09-13T13:32:55.361 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.361 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.361 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3190262686' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:55.361 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:55.362 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:32:55.362 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:54 smithi189 ceph-mon[32912]: pgmap v223: 292 pgs: 64 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:32:55.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:32:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:32:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3190262686' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: osdmap e184: 8 total, 8 up, 8 in 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3190262686' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: osdmap e184: 8 total, 8 up, 8 in 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:55 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3190262686' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:32:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: osdmap e184: 8 total, 8 up, 8 in 2024-09-13T13:32:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/257235334' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:56.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"} : dispatch 2024-09-13T13:32:56.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"} : dispatch 2024-09-13T13:32:56.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:55 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"}]': finished 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: osdmap e185: 8 total, 8 up, 8 in 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/676241986' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: pgmap v226: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/676241986' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: osdmap e186: 8 total, 8 up, 8 in 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"}]': finished 2024-09-13T13:32:57.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: osdmap e185: 8 total, 8 up, 8 in 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/676241986' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: pgmap v226: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/676241986' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: osdmap e186: 8 total, 8 up, 8 in 2024-09-13T13:32:57.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-76315-32"}]': finished 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-21"}]': finished 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: osdmap e185: 8 total, 8 up, 8 in 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/676241986' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: pgmap v226: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-13T13:32:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/676241986' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-76332-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:57.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-76315-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:32:57.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: osdmap e186: 8 total, 8 up, 8 in 2024-09-13T13:32:57.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:58.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[40255]: osdmap e187: 8 total, 8 up, 8 in 2024-09-13T13:32:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[30106]: osdmap e187: 8 total, 8 up, 8 in 2024-09-13T13:32:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:32:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:57 smithi189 ceph-mon[32912]: osdmap e187: 8 total, 8 up, 8 in 2024-09-13T13:32:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:32:59.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[40255]: pgmap v229: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:32:59.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:32:59.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:59.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[40255]: osdmap e188: 8 total, 8 up, 8 in 2024-09-13T13:32:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[30106]: pgmap v229: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:32:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:32:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:32:58 smithi043 ceph-mon[30106]: osdmap e188: 8 total, 8 up, 8 in 2024-09-13T13:32:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:58 smithi189 ceph-mon[32912]: pgmap v229: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:32:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-76315-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:32:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:32:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:32:58 smithi189 ceph-mon[32912]: osdmap e188: 8 total, 8 up, 8 in 2024-09-13T13:33:01.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[40255]: osdmap e189: 8 total, 8 up, 8 in 2024-09-13T13:33:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[40255]: pgmap v232: 236 pgs: 236 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-13T13:33:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[30106]: osdmap e189: 8 total, 8 up, 8 in 2024-09-13T13:33:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:00 smithi043 ceph-mon[30106]: pgmap v232: 236 pgs: 236 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-13T13:33:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:00 smithi189 ceph-mon[32912]: osdmap e189: 8 total, 8 up, 8 in 2024-09-13T13:33:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:00 smithi189 ceph-mon[32912]: pgmap v232: 236 pgs: 236 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-13T13:33:02.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: osdmap e190: 8 total, 8 up, 8 in 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3986274413' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: osdmap e190: 8 total, 8 up, 8 in 2024-09-13T13:33:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3986274413' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:01 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: osdmap e190: 8 total, 8 up, 8 in 2024-09-13T13:33:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3986274413' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:01 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"}]': finished 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3986274413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: osdmap e191: 8 total, 8 up, 8 in 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"} : dispatch 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"} : dispatch 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: pgmap v235: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"}]': finished 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: osdmap e192: 8 total, 8 up, 8 in 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/427977369' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"}]': finished 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3986274413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:03.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: osdmap e191: 8 total, 8 up, 8 in 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"} : dispatch 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"} : dispatch 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: pgmap v235: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"}]': finished 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: osdmap e192: 8 total, 8 up, 8 in 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/427977369' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:03.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-23"}]': finished 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3986274413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: osdmap e191: 8 total, 8 up, 8 in 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"} : dispatch 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"} : dispatch 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/447783353' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"} : dispatch 2024-09-13T13:33:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: pgmap v235: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:33:03.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:03.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-23", "mode": "writeback"}]': finished 2024-09-13T13:33:03.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-76315-33"}]': finished 2024-09-13T13:33:03.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: osdmap e192: 8 total, 8 up, 8 in 2024-09-13T13:33:03.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/427977369' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:03.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:04.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-76315-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-76315-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[30106]: osdmap e193: 8 total, 8 up, 8 in 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi043-76315-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-76315-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-76315-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[40255]: osdmap e193: 8 total, 8 up, 8 in 2024-09-13T13:33:04.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi043-76315-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-76315-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-76315-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:03 smithi189 ceph-mon[32912]: osdmap e193: 8 total, 8 up, 8 in 2024-09-13T13:33:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi043-76315-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:05.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: pgmap v238: 292 pgs: 64 unknown, 228 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: osdmap e194: 8 total, 8 up, 8 in 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2839941075' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: pgmap v238: 292 pgs: 64 unknown, 228 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: osdmap e194: 8 total, 8 up, 8 in 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2839941075' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: pgmap v238: 292 pgs: 64 unknown, 228 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: osdmap e194: 8 total, 8 up, 8 in 2024-09-13T13:33:05.366 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"} : dispatch 2024-09-13T13:33:05.367 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2839941075' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:05.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:33:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:33:05.980 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:33:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:33:06.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:06.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-76315-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"}]': finished 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2839941075' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[30106]: osdmap e195: 8 total, 8 up, 8 in 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-76315-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"}]': finished 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2839941075' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:05 smithi043 ceph-mon[40255]: osdmap e195: 8 total, 8 up, 8 in 2024-09-13T13:33:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:05 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:05 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-76315-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-23"}]': finished 2024-09-13T13:33:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2839941075' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:05 smithi189 ceph-mon[32912]: osdmap e195: 8 total, 8 up, 8 in 2024-09-13T13:33:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[30106]: pgmap v241: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T13:33:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[30106]: osdmap e196: 8 total, 8 up, 8 in 2024-09-13T13:33:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1273825367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[40255]: pgmap v241: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T13:33:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[40255]: osdmap e196: 8 total, 8 up, 8 in 2024-09-13T13:33:07.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1273825367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:06 smithi189 ceph-mon[32912]: pgmap v241: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-13T13:33:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:06 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:06 smithi189 ceph-mon[32912]: osdmap e196: 8 total, 8 up, 8 in 2024-09-13T13:33:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1273825367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1273825367' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[30106]: osdmap e197: 8 total, 8 up, 8 in 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[30106]: pgmap v244: 356 pgs: 64 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1273825367' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[40255]: osdmap e197: 8 total, 8 up, 8 in 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:09.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:09.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:08 smithi043 ceph-mon[40255]: pgmap v244: 356 pgs: 64 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:33:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1273825367' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:08 smithi189 ceph-mon[32912]: osdmap e197: 8 total, 8 up, 8 in 2024-09-13T13:33:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:08 smithi189 ceph-mon[32912]: pgmap v244: 356 pgs: 64 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:33:10.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[30106]: osdmap e198: 8 total, 8 up, 8 in 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/750946614' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[40255]: osdmap e198: 8 total, 8 up, 8 in 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/750946614' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:09 smithi189 ceph-mon[32912]: osdmap e198: 8 total, 8 up, 8 in 2024-09-13T13:33:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"} : dispatch 2024-09-13T13:33:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/750946614' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/750946614' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: osdmap e199: 8 total, 8 up, 8 in 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[30106]: pgmap v247: 388 pgs: 388 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/750946614' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: osdmap e199: 8 total, 8 up, 8 in 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:11.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:10 smithi043 ceph-mon[40255]: pgmap v247: 388 pgs: 388 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/933193729' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-76315-34"}]': finished 2024-09-13T13:33:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/750946614' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-76332-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: osdmap e199: 8 total, 8 up, 8 in 2024-09-13T13:33:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:10 smithi189 ceph-mon[32912]: pgmap v247: 388 pgs: 388 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: osdmap e200: 8 total, 8 up, 8 in 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"}]': finished 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"} : dispatch 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: osdmap e201: 8 total, 8 up, 8 in 2024-09-13T13:33:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"} : dispatch 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: osdmap e200: 8 total, 8 up, 8 in 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"}]': finished 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"} : dispatch 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: osdmap e201: 8 total, 8 up, 8 in 2024-09-13T13:33:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"} : dispatch 2024-09-13T13:33:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-76315-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: osdmap e200: 8 total, 8 up, 8 in 2024-09-13T13:33:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-25"}]': finished 2024-09-13T13:33:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"} : dispatch 2024-09-13T13:33:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: osdmap e201: 8 total, 8 up, 8 in 2024-09-13T13:33:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"} : dispatch 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[30106]: pgmap v250: 324 pgs: 324 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"}]': finished 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[30106]: osdmap e202: 8 total, 8 up, 8 in 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[40255]: pgmap v250: 324 pgs: 324 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:33:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"}]': finished 2024-09-13T13:33:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[40255]: osdmap e202: 8 total, 8 up, 8 in 2024-09-13T13:33:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:12 smithi189 ceph-mon[32912]: pgmap v250: 324 pgs: 324 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:33:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:12 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-76315-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-25", "mode": "writeback"}]': finished 2024-09-13T13:33:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:12 smithi189 ceph-mon[32912]: osdmap e202: 8 total, 8 up, 8 in 2024-09-13T13:33:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:14 smithi189 ceph-mon[32912]: osdmap e203: 8 total, 8 up, 8 in 2024-09-13T13:33:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:14 smithi189 ceph-mon[32912]: pgmap v253: 268 pgs: 8 unknown, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:15.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[30106]: osdmap e203: 8 total, 8 up, 8 in 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[30106]: pgmap v253: 268 pgs: 8 unknown, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[40255]: osdmap e203: 8 total, 8 up, 8 in 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"} : dispatch 2024-09-13T13:33:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:14 smithi043 ceph-mon[40255]: pgmap v253: 268 pgs: 8 unknown, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:15.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:33:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:33:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:33:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"}]': finished 2024-09-13T13:33:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[30106]: osdmap e204: 8 total, 8 up, 8 in 2024-09-13T13:33:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"}]': finished 2024-09-13T13:33:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[40255]: osdmap e204: 8 total, 8 up, 8 in 2024-09-13T13:33:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:15 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-25"}]': finished 2024-09-13T13:33:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-mon[32912]: osdmap e204: 8 total, 8 up, 8 in 2024-09-13T13:33:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:16.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:16.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:15 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: osdmap e205: 8 total, 8 up, 8 in 2024-09-13T13:33:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2446748740' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: pgmap v256: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: osdmap e206: 8 total, 8 up, 8 in 2024-09-13T13:33:17.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:17.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: osdmap e205: 8 total, 8 up, 8 in 2024-09-13T13:33:17.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:17.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:17.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2446748740' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: pgmap v256: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:17.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:17.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:17.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: osdmap e206: 8 total, 8 up, 8 in 2024-09-13T13:33:17.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: osdmap e205: 8 total, 8 up, 8 in 2024-09-13T13:33:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3207310904' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"} : dispatch 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2446748740' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: pgmap v256: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-76315-35"}]': finished 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-76332-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: osdmap e206: 8 total, 8 up, 8 in 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: osdmap e207: 8 total, 8 up, 8 in 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: osdmap e207: 8 total, 8 up, 8 in 2024-09-13T13:33:18.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-76315-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: osdmap e207: 8 total, 8 up, 8 in 2024-09-13T13:33:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[40255]: pgmap v259: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[40255]: osdmap e208: 8 total, 8 up, 8 in 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1764720278' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[30106]: pgmap v259: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[30106]: osdmap e208: 8 total, 8 up, 8 in 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1764720278' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:19.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:18 smithi189 ceph-mon[32912]: pgmap v259: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:18 smithi189 ceph-mon[32912]: osdmap e208: 8 total, 8 up, 8 in 2024-09-13T13:33:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1764720278' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:21.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:21.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:21.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:21.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:21.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:21.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: osdmap e209: 8 total, 8 up, 8 in 2024-09-13T13:33:21.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:21.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:21.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:20 smithi189 ceph-mon[32912]: pgmap v262: 268 pgs: 8 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: osdmap e209: 8 total, 8 up, 8 in 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[30106]: pgmap v262: 268 pgs: 8 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-76315-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-76332-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: osdmap e209: 8 total, 8 up, 8 in 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:20 smithi043 ceph-mon[40255]: pgmap v262: 268 pgs: 8 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-13T13:33:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"}]': finished 2024-09-13T13:33:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: osdmap e210: 8 total, 8 up, 8 in 2024-09-13T13:33:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"} : dispatch 2024-09-13T13:33:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"} : dispatch 2024-09-13T13:33:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:22.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"}]': finished 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: osdmap e210: 8 total, 8 up, 8 in 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"} : dispatch 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"} : dispatch 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-27"}]': finished 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: osdmap e210: 8 total, 8 up, 8 in 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"} : dispatch 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"} : dispatch 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:22.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"}]': finished 2024-09-13T13:33:23.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:23.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: osdmap e211: 8 total, 8 up, 8 in 2024-09-13T13:33:23.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:23.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:23.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:23.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:23.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:22 smithi189 ceph-mon[32912]: pgmap v265: 228 pgs: 228 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"}]': finished 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: osdmap e211: 8 total, 8 up, 8 in 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[30106]: pgmap v265: 228 pgs: 228 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-27", "mode": "writeback"}]': finished 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: osdmap e211: 8 total, 8 up, 8 in 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:23.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:22 smithi043 ceph-mon[40255]: pgmap v265: 228 pgs: 228 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:33:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[30106]: osdmap e212: 8 total, 8 up, 8 in 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[40255]: osdmap e212: 8 total, 8 up, 8 in 2024-09-13T13:33:24.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:24.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:24.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1195427060' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:23 smithi189 ceph-mon[32912]: osdmap e212: 8 total, 8 up, 8 in 2024-09-13T13:33:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"} : dispatch 2024-09-13T13:33:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: osdmap e213: 8 total, 8 up, 8 in 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: pgmap v268: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"}]': finished 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: osdmap e214: 8 total, 8 up, 8 in 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-76315-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: osdmap e213: 8 total, 8 up, 8 in 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:25.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: pgmap v268: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"}]': finished 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: osdmap e214: 8 total, 8 up, 8 in 2024-09-13T13:33:25.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-76315-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-76315-36"}]': finished 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: osdmap e213: 8 total, 8 up, 8 in 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"} : dispatch 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.383 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.384 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:25.384 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: pgmap v268: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:25.384 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:25.384 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-27"}]': finished 2024-09-13T13:33:25.384 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:25.384 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: osdmap e214: 8 total, 8 up, 8 in 2024-09-13T13:33:25.384 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-76315-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:25.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:33:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:33:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:25 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:25 smithi043 ceph-mon[30106]: osdmap e215: 8 total, 8 up, 8 in 2024-09-13T13:33:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:25 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:25 smithi043 ceph-mon[40255]: osdmap e215: 8 total, 8 up, 8 in 2024-09-13T13:33:25.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:33:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:33:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:33:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:25 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:25 smithi189 ceph-mon[32912]: osdmap e215: 8 total, 8 up, 8 in 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: pgmap v271: 196 pgs: 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-76315-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: osdmap e216: 8 total, 8 up, 8 in 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: pgmap v271: 196 pgs: 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-76315-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: osdmap e216: 8 total, 8 up, 8 in 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: pgmap v271: 196 pgs: 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-76315-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: osdmap e216: 8 total, 8 up, 8 in 2024-09-13T13:33:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1833535028' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"} : dispatch 2024-09-13T13:33:28.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:28.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[30106]: osdmap e217: 8 total, 8 up, 8 in 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:27 smithi043 ceph-mon[40255]: osdmap e217: 8 total, 8 up, 8 in 2024-09-13T13:33:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:27 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-46"}]': finished 2024-09-13T13:33:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:27 smithi189 ceph-mon[32912]: osdmap e217: 8 total, 8 up, 8 in 2024-09-13T13:33:29.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: pgmap v274: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: osdmap e218: 8 total, 8 up, 8 in 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: pgmap v274: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: osdmap e218: 8 total, 8 up, 8 in 2024-09-13T13:33:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: pgmap v274: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-76332-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: osdmap e218: 8 total, 8 up, 8 in 2024-09-13T13:33:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[30106]: osdmap e219: 8 total, 8 up, 8 in 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[30106]: pgmap v277: 228 pgs: 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[40255]: osdmap e219: 8 total, 8 up, 8 in 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:30 smithi043 ceph-mon[40255]: pgmap v277: 228 pgs: 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:30 smithi189 ceph-mon[32912]: osdmap e219: 8 total, 8 up, 8 in 2024-09-13T13:33:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"} : dispatch 2024-09-13T13:33:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:31.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:30 smithi189 ceph-mon[32912]: pgmap v277: 228 pgs: 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: osdmap e220: 8 total, 8 up, 8 in 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"}]': finished 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: osdmap e221: 8 total, 8 up, 8 in 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"} : dispatch 2024-09-13T13:33:32.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: osdmap e220: 8 total, 8 up, 8 in 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:32.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"}]': finished 2024-09-13T13:33:32.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:32.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: osdmap e221: 8 total, 8 up, 8 in 2024-09-13T13:33:32.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"} : dispatch 2024-09-13T13:33:32.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"} : dispatch 2024-09-13T13:33:32.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-76332-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1182057967' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-76315-37"}]': finished 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: osdmap e220: 8 total, 8 up, 8 in 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:32.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-29"}]': finished 2024-09-13T13:33:32.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-76315-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:32.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: osdmap e221: 8 total, 8 up, 8 in 2024-09-13T13:33:32.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"} : dispatch 2024-09-13T13:33:32.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:32.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"} : dispatch 2024-09-13T13:33:32.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[30106]: pgmap v280: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"}]': finished 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[30106]: osdmap e222: 8 total, 8 up, 8 in 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[40255]: pgmap v280: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"}]': finished 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[40255]: osdmap e222: 8 total, 8 up, 8 in 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:32 smithi189 ceph-mon[32912]: pgmap v280: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:32 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-29", "mode": "writeback"}]': finished 2024-09-13T13:33:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:32 smithi189 ceph-mon[32912]: osdmap e222: 8 total, 8 up, 8 in 2024-09-13T13:33:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:34.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[30106]: osdmap e223: 8 total, 8 up, 8 in 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[40255]: osdmap e223: 8 total, 8 up, 8 in 2024-09-13T13:33:34.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:34.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:34.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:33 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:34.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-76315-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:34.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:34.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:33 smithi189 ceph-mon[32912]: osdmap e223: 8 total, 8 up, 8 in 2024-09-13T13:33:34.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3000064691' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:34.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"} : dispatch 2024-09-13T13:33:35.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: pgmap v283: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: osdmap e224: 8 total, 8 up, 8 in 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:35.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: pgmap v283: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: osdmap e224: 8 total, 8 up, 8 in 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:35.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: pgmap v283: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:35.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-76332-47"}]': finished 2024-09-13T13:33:35.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: osdmap e224: 8 total, 8 up, 8 in 2024-09-13T13:33:35.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:35.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:35.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:35.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:35.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:33:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:33:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: osdmap e225: 8 total, 8 up, 8 in 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: osdmap e225: 8 total, 8 up, 8 in 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:35.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:35.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:35.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:35.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-76332-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: osdmap e225: 8 total, 8 up, 8 in 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"} : dispatch 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[40255]: pgmap v286: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-13T13:33:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"}]': finished 2024-09-13T13:33:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[40255]: osdmap e226: 8 total, 8 up, 8 in 2024-09-13T13:33:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[30106]: pgmap v286: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-13T13:33:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"}]': finished 2024-09-13T13:33:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[30106]: osdmap e226: 8 total, 8 up, 8 in 2024-09-13T13:33:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:36 smithi189 ceph-mon[32912]: pgmap v286: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-13T13:33:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:36 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-29"}]': finished 2024-09-13T13:33:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/511972189' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:36 smithi189 ceph-mon[32912]: osdmap e226: 8 total, 8 up, 8 in 2024-09-13T13:33:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"} : dispatch 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: osdmap e227: 8 total, 8 up, 8 in 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[30106]: pgmap v289: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: osdmap e227: 8 total, 8 up, 8 in 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:39.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:38 smithi043 ceph-mon[40255]: pgmap v289: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-76332-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-76315-38"}]': finished 2024-09-13T13:33:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: osdmap e227: 8 total, 8 up, 8 in 2024-09-13T13:33:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:38 smithi189 ceph-mon[32912]: pgmap v289: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-13T13:33:40.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[30106]: osdmap e228: 8 total, 8 up, 8 in 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[40255]: osdmap e228: 8 total, 8 up, 8 in 2024-09-13T13:33:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:40.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:39 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-76315-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:39 smithi189 ceph-mon[32912]: osdmap e228: 8 total, 8 up, 8 in 2024-09-13T13:33:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[30106]: osdmap e229: 8 total, 8 up, 8 in 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[30106]: pgmap v292: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:41.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[40255]: osdmap e229: 8 total, 8 up, 8 in 2024-09-13T13:33:41.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:41.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:41.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:41.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:40 smithi043 ceph-mon[40255]: pgmap v292: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:40 smithi189 ceph-mon[32912]: osdmap e229: 8 total, 8 up, 8 in 2024-09-13T13:33:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:41.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:40 smithi189 ceph-mon[32912]: pgmap v292: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: osdmap e230: 8 total, 8 up, 8 in 2024-09-13T13:33:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"}]': finished 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: osdmap e231: 8 total, 8 up, 8 in 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:42.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: osdmap e230: 8 total, 8 up, 8 in 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"}]': finished 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:42.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: osdmap e231: 8 total, 8 up, 8 in 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"} : dispatch 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"} : dispatch 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:42.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-76315-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: osdmap e230: 8 total, 8 up, 8 in 2024-09-13T13:33:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/567886483' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-31"}]': finished 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-76332-48"}]': finished 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: osdmap e231: 8 total, 8 up, 8 in 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: pgmap v295: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"}]': finished 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: osdmap e232: 8 total, 8 up, 8 in 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: pgmap v295: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"}]': finished 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: osdmap e232: 8 total, 8 up, 8 in 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: pgmap v295: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-31", "mode": "writeback"}]': finished 2024-09-13T13:33:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-76332-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: osdmap e232: 8 total, 8 up, 8 in 2024-09-13T13:33:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:43.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:43.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:43.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:44.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: osdmap e233: 8 total, 8 up, 8 in 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: osdmap e233: 8 total, 8 up, 8 in 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:44.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3922522949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: osdmap e233: 8 total, 8 up, 8 in 2024-09-13T13:33:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"} : dispatch 2024-09-13T13:33:44.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"} : dispatch 2024-09-13T13:33:45.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: pgmap v298: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"}]': finished 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: osdmap e234: 8 total, 8 up, 8 in 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: pgmap v298: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"}]': finished 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: osdmap e234: 8 total, 8 up, 8 in 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: pgmap v298: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-76332-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-76315-39"}]': finished 2024-09-13T13:33:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-31"}]': finished 2024-09-13T13:33:45.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: osdmap e234: 8 total, 8 up, 8 in 2024-09-13T13:33:45.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:45.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:45.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:33:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:33:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[40255]: osdmap e235: 8 total, 8 up, 8 in 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:33:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[30106]: osdmap e235: 8 total, 8 up, 8 in 2024-09-13T13:33:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:33:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-76315-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-mon[32912]: osdmap e235: 8 total, 8 up, 8 in 2024-09-13T13:33:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:46.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:47.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:33:47.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: pgmap v301: 204 pgs: 8 creating+peering, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: osdmap e236: 8 total, 8 up, 8 in 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: pgmap v301: 204 pgs: 8 creating+peering, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: osdmap e236: 8 total, 8 up, 8 in 2024-09-13T13:33:47.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:47.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:47.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:47.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:33:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:33:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: pgmap v301: 204 pgs: 8 creating+peering, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: osdmap e236: 8 total, 8 up, 8 in 2024-09-13T13:33:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:49.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: osdmap e237: 8 total, 8 up, 8 in 2024-09-13T13:33:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:49.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:49.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:49.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[30106]: pgmap v304: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:49.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: osdmap e237: 8 total, 8 up, 8 in 2024-09-13T13:33:49.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:49.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:49.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:49.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:49.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:48 smithi043 ceph-mon[40255]: pgmap v304: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-76315-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: osdmap e237: 8 total, 8 up, 8 in 2024-09-13T13:33:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2265185193' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"} : dispatch 2024-09-13T13:33:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:49.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:48 smithi189 ceph-mon[32912]: pgmap v304: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:50.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:50.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:50.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: osdmap e238: 8 total, 8 up, 8 in 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-76332-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: osdmap e238: 8 total, 8 up, 8 in 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:50.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-76332-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-76332-49"}]': finished 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: osdmap e238: 8 total, 8 up, 8 in 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-76332-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"}]': finished 2024-09-13T13:33:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-76332-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: osdmap e239: 8 total, 8 up, 8 in 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"} : dispatch 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-76332-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"} : dispatch 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: pgmap v307: 228 pgs: 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"}]': finished 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"}]': finished 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-76332-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: osdmap e239: 8 total, 8 up, 8 in 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"} : dispatch 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-76332-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"} : dispatch 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: pgmap v307: 228 pgs: 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:51.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"}]': finished 2024-09-13T13:33:51.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-33"}]': finished 2024-09-13T13:33:51.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-76332-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: osdmap e239: 8 total, 8 up, 8 in 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"} : dispatch 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-76332-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"} : dispatch 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: pgmap v307: 228 pgs: 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-33", "mode": "writeback"}]': finished 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: osdmap e240: 8 total, 8 up, 8 in 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-76332-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: osdmap e241: 8 total, 8 up, 8 in 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: osdmap e240: 8 total, 8 up, 8 in 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-76332-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:52.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: osdmap e241: 8 total, 8 up, 8 in 2024-09-13T13:33:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1308727818' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: osdmap e240: 8 total, 8 up, 8 in 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"} : dispatch 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-76332-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-76315-40"}]': finished 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: osdmap e241: 8 total, 8 up, 8 in 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:52.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"} : dispatch 2024-09-13T13:33:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: pgmap v310: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:53.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"}]': finished 2024-09-13T13:33:53.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:53.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: osdmap e242: 8 total, 8 up, 8 in 2024-09-13T13:33:53.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:53.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:53.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: pgmap v310: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:53.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:53.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"}]': finished 2024-09-13T13:33:53.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:53.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: osdmap e242: 8 total, 8 up, 8 in 2024-09-13T13:33:53.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: pgmap v310: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-33"}]': finished 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-76315-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: osdmap e242: 8 total, 8 up, 8 in 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:53.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:54 smithi043 ceph-mon[40255]: osdmap e243: 8 total, 8 up, 8 in 2024-09-13T13:33:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:54 smithi043 ceph-mon[40255]: pgmap v313: 196 pgs: 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:54 smithi043 ceph-mon[30106]: osdmap e243: 8 total, 8 up, 8 in 2024-09-13T13:33:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:54 smithi043 ceph-mon[30106]: pgmap v313: 196 pgs: 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:54 smithi189 ceph-mon[32912]: osdmap e243: 8 total, 8 up, 8 in 2024-09-13T13:33:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:54 smithi189 ceph-mon[32912]: pgmap v313: 196 pgs: 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:33:55.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:33:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:33:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:33:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[40255]: osdmap e244: 8 total, 8 up, 8 in 2024-09-13T13:33:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[30106]: osdmap e244: 8 total, 8 up, 8 in 2024-09-13T13:33:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:55 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-76315-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-mon[32912]: osdmap e244: 8 total, 8 up, 8 in 2024-09-13T13:33:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"} : dispatch 2024-09-13T13:33:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:56.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:55 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:33:57.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:57.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[40255]: osdmap e245: 8 total, 8 up, 8 in 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[40255]: pgmap v316: 236 pgs: 40 creating+peering, 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[30106]: osdmap e245: 8 total, 8 up, 8 in 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:56 smithi043 ceph-mon[30106]: pgmap v316: 236 pgs: 40 creating+peering, 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1957659518' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-76332-50"}]': finished 2024-09-13T13:33:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:56 smithi189 ceph-mon[32912]: osdmap e245: 8 total, 8 up, 8 in 2024-09-13T13:33:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:33:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:56 smithi189 ceph-mon[32912]: pgmap v316: 236 pgs: 40 creating+peering, 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:58.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: osdmap e246: 8 total, 8 up, 8 in 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3246660245' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: osdmap e246: 8 total, 8 up, 8 in 2024-09-13T13:33:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:33:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3246660245' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:33:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:33:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:33:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: osdmap e246: 8 total, 8 up, 8 in 2024-09-13T13:33:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:33:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3246660245' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:33:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:33:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3246660245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: osdmap e247: 8 total, 8 up, 8 in 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"} : dispatch 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"} : dispatch 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: pgmap v319: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"}]': finished 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[30106]: osdmap e248: 8 total, 8 up, 8 in 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3246660245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: osdmap e247: 8 total, 8 up, 8 in 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"} : dispatch 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"} : dispatch 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:59.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: pgmap v319: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:59.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:59.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"}]': finished 2024-09-13T13:33:59.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:59.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:58 smithi043 ceph-mon[40255]: osdmap e248: 8 total, 8 up, 8 in 2024-09-13T13:33:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3246660245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-76332-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:33:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:33:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: osdmap e247: 8 total, 8 up, 8 in 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"} : dispatch 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1842136026' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"} : dispatch 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"} : dispatch 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: pgmap v319: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-35", "mode": "writeback"}]': finished 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-76315-41"}]': finished 2024-09-13T13:33:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:58 smithi189 ceph-mon[32912]: osdmap e248: 8 total, 8 up, 8 in 2024-09-13T13:34:00.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: osdmap e249: 8 total, 8 up, 8 in 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:00.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:00.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: osdmap e249: 8 total, 8 up, 8 in 2024-09-13T13:34:00.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:33:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-76315-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: osdmap e249: 8 total, 8 up, 8 in 2024-09-13T13:34:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:00.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:00.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:33:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:01.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:00 smithi043 ceph-mon[30106]: pgmap v322: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:34:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:00 smithi043 ceph-mon[40255]: pgmap v322: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:34:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:00 smithi189 ceph-mon[32912]: pgmap v322: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:34:02.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: osdmap e250: 8 total, 8 up, 8 in 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: osdmap e251: 8 total, 8 up, 8 in 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: osdmap e250: 8 total, 8 up, 8 in 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: osdmap e251: 8 total, 8 up, 8 in 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: osdmap e250: 8 total, 8 up, 8 in 2024-09-13T13:34:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-76315-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-76332-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: osdmap e251: 8 total, 8 up, 8 in 2024-09-13T13:34:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:02 smithi043 ceph-mon[30106]: pgmap v325: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:34:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:34:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:02 smithi043 ceph-mon[30106]: osdmap e252: 8 total, 8 up, 8 in 2024-09-13T13:34:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:02 smithi043 ceph-mon[40255]: pgmap v325: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:34:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:34:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:02 smithi043 ceph-mon[40255]: osdmap e252: 8 total, 8 up, 8 in 2024-09-13T13:34:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:02 smithi189 ceph-mon[32912]: pgmap v325: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:34:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:34:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:02 smithi189 ceph-mon[32912]: osdmap e252: 8 total, 8 up, 8 in 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[30106]: osdmap e253: 8 total, 8 up, 8 in 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[40255]: osdmap e253: 8 total, 8 up, 8 in 2024-09-13T13:34:04.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:04.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:04.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:04.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:03 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:03 smithi189 ceph-mon[32912]: osdmap e253: 8 total, 8 up, 8 in 2024-09-13T13:34:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: pgmap v328: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: osdmap e254: 8 total, 8 up, 8 in 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: pgmap v328: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: osdmap e254: 8 total, 8 up, 8 in 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:05.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:05.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: pgmap v328: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1245369977' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: osdmap e254: 8 total, 8 up, 8 in 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"} : dispatch 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4124679391' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"} : dispatch 2024-09-13T13:34:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"} : dispatch 2024-09-13T13:34:05.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:34:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7014 ms) 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6947 ms) 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6992 ms) 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7010 ms) 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7026 ms) 2024-09-13T13:34:05.965 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7014 ms) 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7019 ms) 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7021 ms) 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6920 ms) 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7019 ms) 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7023 ms) 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6972 ms) 2024-09-13T13:34:05.966 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6985 ms) 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7016 ms) 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7028 ms) 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7024 ms) 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112030 ms total) 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment tear-down 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (189404 ms total) 2024-09-13T13:34:05.967 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ PASSED ] 42 tests. 2024-09-13T13:34:05.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:34:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:34:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:06.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:06.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:06.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:34:06.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:06.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[40255]: osdmap e255: 8 total, 8 up, 8 in 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:05 smithi043 ceph-mon[30106]: osdmap e255: 8 total, 8 up, 8 in 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:05 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:06.545 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:05 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:06.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-76332-52"}]': finished 2024-09-13T13:34:06.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-35"}]': finished 2024-09-13T13:34:06.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-76315-42"}]': finished 2024-09-13T13:34:06.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:05 smithi189 ceph-mon[32912]: osdmap e255: 8 total, 8 up, 8 in 2024-09-13T13:34:07.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[30106]: pgmap v331: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-13T13:34:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:06 smithi043 ceph-mon[40255]: pgmap v331: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-13T13:34:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:06 smithi189 ceph-mon[32912]: pgmap v331: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-13T13:34:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:08 smithi189 ceph-mon[32912]: osdmap e256: 8 total, 8 up, 8 in 2024-09-13T13:34:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-76332-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:08 smithi043 ceph-mon[30106]: osdmap e256: 8 total, 8 up, 8 in 2024-09-13T13:34:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-76332-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:08 smithi043 ceph-mon[40255]: osdmap e256: 8 total, 8 up, 8 in 2024-09-13T13:34:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-76332-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: osdmap e257: 8 total, 8 up, 8 in 2024-09-13T13:34:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: pgmap v334: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:34:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-76332-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: osdmap e258: 8 total, 8 up, 8 in 2024-09-13T13:34:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:09.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: osdmap e257: 8 total, 8 up, 8 in 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: pgmap v334: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-76332-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: osdmap e258: 8 total, 8 up, 8 in 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: osdmap e257: 8 total, 8 up, 8 in 2024-09-13T13:34:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: pgmap v334: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:34:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-76332-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: osdmap e258: 8 total, 8 up, 8 in 2024-09-13T13:34:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:10 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:10 smithi189 ceph-mon[32912]: osdmap e259: 8 total, 8 up, 8 in 2024-09-13T13:34:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[30106]: osdmap e259: 8 total, 8 up, 8 in 2024-09-13T13:34:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:10.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[40255]: osdmap e259: 8 total, 8 up, 8 in 2024-09-13T13:34:10.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:10.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:11 smithi189 ceph-mon[32912]: pgmap v337: 236 pgs: 236 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 op/s 2024-09-13T13:34:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"}]': finished 2024-09-13T13:34:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:11 smithi189 ceph-mon[32912]: osdmap e260: 8 total, 8 up, 8 in 2024-09-13T13:34:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"} : dispatch 2024-09-13T13:34:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"} : dispatch 2024-09-13T13:34:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[40255]: pgmap v337: 236 pgs: 236 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 op/s 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"}]': finished 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[40255]: osdmap e260: 8 total, 8 up, 8 in 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"} : dispatch 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"} : dispatch 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[30106]: pgmap v337: 236 pgs: 236 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 op/s 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-37"}]': finished 2024-09-13T13:34:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[30106]: osdmap e260: 8 total, 8 up, 8 in 2024-09-13T13:34:11.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:11.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"} : dispatch 2024-09-13T13:34:11.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"} : dispatch 2024-09-13T13:34:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:12 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:34:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"}]': finished 2024-09-13T13:34:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:12 smithi189 ceph-mon[32912]: osdmap e261: 8 total, 8 up, 8 in 2024-09-13T13:34:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"}]': finished 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[30106]: osdmap e261: 8 total, 8 up, 8 in 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:12.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:34:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-37", "mode": "writeback"}]': finished 2024-09-13T13:34:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[40255]: osdmap e261: 8 total, 8 up, 8 in 2024-09-13T13:34:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"} : dispatch 2024-09-13T13:34:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: pgmap v340: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:34:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: osdmap e262: 8 total, 8 up, 8 in 2024-09-13T13:34:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: pgmap v340: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: osdmap e262: 8 total, 8 up, 8 in 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: pgmap v340: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/15394382' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-76332-53"}]': finished 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: osdmap e262: 8 total, 8 up, 8 in 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"} : dispatch 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:13.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:13.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:14 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"}]': finished 2024-09-13T13:34:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:14 smithi189 ceph-mon[32912]: osdmap e263: 8 total, 8 up, 8 in 2024-09-13T13:34:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"}]': finished 2024-09-13T13:34:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[30106]: osdmap e263: 8 total, 8 up, 8 in 2024-09-13T13:34:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-37"}]': finished 2024-09-13T13:34:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-76332-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[40255]: osdmap e263: 8 total, 8 up, 8 in 2024-09-13T13:34:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:15 smithi189 ceph-mon[32912]: pgmap v343: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:15 smithi189 ceph-mon[32912]: osdmap e264: 8 total, 8 up, 8 in 2024-09-13T13:34:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:15 smithi043 ceph-mon[40255]: pgmap v343: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:15 smithi043 ceph-mon[40255]: osdmap e264: 8 total, 8 up, 8 in 2024-09-13T13:34:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:15 smithi043 ceph-mon[30106]: pgmap v343: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:15 smithi043 ceph-mon[30106]: osdmap e264: 8 total, 8 up, 8 in 2024-09-13T13:34:15.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:34:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:34:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:34:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:34:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:34:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:16 smithi189 ceph-mon[32912]: osdmap e265: 8 total, 8 up, 8 in 2024-09-13T13:34:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:16.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[40255]: osdmap e265: 8 total, 8 up, 8 in 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-76332-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[30106]: osdmap e265: 8 total, 8 up, 8 in 2024-09-13T13:34:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:16.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:17 smithi189 ceph-mon[32912]: pgmap v346: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:17 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:17 smithi189 ceph-mon[32912]: osdmap e266: 8 total, 8 up, 8 in 2024-09-13T13:34:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[30106]: pgmap v346: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[30106]: osdmap e266: 8 total, 8 up, 8 in 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[40255]: pgmap v346: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[40255]: osdmap e266: 8 total, 8 up, 8 in 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[30106]: osdmap e267: 8 total, 8 up, 8 in 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[30106]: pgmap v349: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:19.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[40255]: osdmap e267: 8 total, 8 up, 8 in 2024-09-13T13:34:19.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:19.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:19.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:19.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:19.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:18 smithi043 ceph-mon[40255]: pgmap v349: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:18 smithi189 ceph-mon[32912]: osdmap e267: 8 total, 8 up, 8 in 2024-09-13T13:34:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:19.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:19.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:18 smithi189 ceph-mon[32912]: pgmap v349: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:20.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"}]': finished 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[30106]: osdmap e268: 8 total, 8 up, 8 in 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"} : dispatch 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"} : dispatch 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"}]': finished 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[40255]: osdmap e268: 8 total, 8 up, 8 in 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"} : dispatch 2024-09-13T13:34:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:20.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"} : dispatch 2024-09-13T13:34:20.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-39"}]': finished 2024-09-13T13:34:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:19 smithi189 ceph-mon[32912]: osdmap e268: 8 total, 8 up, 8 in 2024-09-13T13:34:20.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"} : dispatch 2024-09-13T13:34:20.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/92317493' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:20.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"} : dispatch 2024-09-13T13:34:20.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"} : dispatch 2024-09-13T13:34:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:34:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"}]': finished 2024-09-13T13:34:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: osdmap e269: 8 total, 8 up, 8 in 2024-09-13T13:34:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[30106]: pgmap v352: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"}]': finished 2024-09-13T13:34:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: osdmap e269: 8 total, 8 up, 8 in 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:21.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:20 smithi043 ceph-mon[40255]: pgmap v352: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-09-13T13:34:21.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-39", "mode": "writeback"}]': finished 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-76332-54"}]': finished 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: osdmap e269: 8 total, 8 up, 8 in 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:20 smithi189 ceph-mon[32912]: pgmap v352: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[30106]: osdmap e270: 8 total, 8 up, 8 in 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:22.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[40255]: osdmap e270: 8 total, 8 up, 8 in 2024-09-13T13:34:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:22.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-76332-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:34:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:21 smithi189 ceph-mon[32912]: osdmap e270: 8 total, 8 up, 8 in 2024-09-13T13:34:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"} : dispatch 2024-09-13T13:34:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:22.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-13T13:34:22.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-13T13:34:22.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-13T13:34:22.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: seed 76627 2024-09-13T13:34:22.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-13T13:34:22.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (258 ms) 2024-09-13T13:34:22.931 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10244 ms) 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (7 ms) 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (10509 ms total) 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7034 ms) 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7958 ms) 2024-09-13T13:34:22.932 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9992 ms) 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [3] 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [4,3] 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [5,4,3] 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting some heads 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-13T13:34:22.933 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 3 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46011 ms) 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9997 ms) 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8018 ms) 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8019 ms) 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8030 ms) 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-13T13:34:22.934 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10027 ms) 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8925 ms) 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10035 ms) 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (11986 ms) 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7982 ms) 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8029 ms) 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13075 ms) 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-13T13:34:22.935 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7908 ms) 2024-09-13T13:34:22.936 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-13T13:34:22.936 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8028 ms) 2024-09-13T13:34:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"}]': finished 2024-09-13T13:34:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[30106]: osdmap e271: 8 total, 8 up, 8 in 2024-09-13T13:34:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[30106]: pgmap v355: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[30106]: osdmap e272: 8 total, 8 up, 8 in 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"}]': finished 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[40255]: osdmap e271: 8 total, 8 up, 8 in 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[40255]: pgmap v355: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:22 smithi043 ceph-mon[40255]: osdmap e272: 8 total, 8 up, 8 in 2024-09-13T13:34:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:22 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:34:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-39"}]': finished 2024-09-13T13:34:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:22 smithi189 ceph-mon[32912]: osdmap e271: 8 total, 8 up, 8 in 2024-09-13T13:34:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:22 smithi189 ceph-mon[32912]: pgmap v355: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 71 op/s 2024-09-13T13:34:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-76332-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:22 smithi189 ceph-mon[32912]: osdmap e272: 8 total, 8 up, 8 in 2024-09-13T13:34:25.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[30106]: osdmap e273: 8 total, 8 up, 8 in 2024-09-13T13:34:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:25.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[30106]: pgmap v358: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[40255]: osdmap e273: 8 total, 8 up, 8 in 2024-09-13T13:34:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:25.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:24 smithi043 ceph-mon[40255]: pgmap v358: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:24 smithi189 ceph-mon[32912]: osdmap e273: 8 total, 8 up, 8 in 2024-09-13T13:34:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:24 smithi189 ceph-mon[32912]: pgmap v358: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:25.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:34:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:34:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: osdmap e274: 8 total, 8 up, 8 in 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: osdmap e274: 8 total, 8 up, 8 in 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:25.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:25 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: osdmap e274: 8 total, 8 up, 8 in 2024-09-13T13:34:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:26.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:25 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: osdmap e275: 8 total, 8 up, 8 in 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[40255]: pgmap v361: 228 pgs: 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: osdmap e275: 8 total, 8 up, 8 in 2024-09-13T13:34:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:34:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:34:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:26 smithi043 ceph-mon[30106]: pgmap v361: 228 pgs: 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: osdmap e275: 8 total, 8 up, 8 in 2024-09-13T13:34:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:34:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4149303460' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:34:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"} : dispatch 2024-09-13T13:34:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:26 smithi189 ceph-mon[32912]: pgmap v361: 228 pgs: 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: osdmap e276: 8 total, 8 up, 8 in 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: osdmap e276: 8 total, 8 up, 8 in 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:28.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:34:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-76332-55"}]': finished 2024-09-13T13:34:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: osdmap e276: 8 total, 8 up, 8 in 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:28.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: osdmap e277: 8 total, 8 up, 8 in 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: pgmap v364: 228 pgs: 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: osdmap e277: 8 total, 8 up, 8 in 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: pgmap v364: 228 pgs: 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:29.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:34:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:34:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-76332-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: osdmap e277: 8 total, 8 up, 8 in 2024-09-13T13:34:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:34:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:34:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: pgmap v364: 228 pgs: 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:34:30.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:29 smithi043 ceph-mon[40255]: osdmap e278: 8 total, 8 up, 8 in 2024-09-13T13:34:30.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:30.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:29 smithi043 ceph-mon[40255]: osdmap e279: 8 total, 8 up, 8 in 2024-09-13T13:34:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:29 smithi043 ceph-mon[30106]: osdmap e278: 8 total, 8 up, 8 in 2024-09-13T13:34:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:29 smithi043 ceph-mon[30106]: osdmap e279: 8 total, 8 up, 8 in 2024-09-13T13:34:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:29 smithi189 ceph-mon[32912]: osdmap e278: 8 total, 8 up, 8 in 2024-09-13T13:34:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-76332-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:29 smithi189 ceph-mon[32912]: osdmap e279: 8 total, 8 up, 8 in 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"} : dispatch 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"} : dispatch 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[30106]: pgmap v367: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"}]': finished 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[30106]: osdmap e280: 8 total, 8 up, 8 in 2024-09-13T13:34:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"} : dispatch 2024-09-13T13:34:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"} : dispatch 2024-09-13T13:34:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[40255]: pgmap v367: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"}]': finished 2024-09-13T13:34:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:30 smithi043 ceph-mon[40255]: osdmap e280: 8 total, 8 up, 8 in 2024-09-13T13:34:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"} : dispatch 2024-09-13T13:34:31.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"} : dispatch 2024-09-13T13:34:31.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:30 smithi189 ceph-mon[32912]: pgmap v367: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:31.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-41"}]': finished 2024-09-13T13:34:31.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:30 smithi189 ceph-mon[32912]: osdmap e280: 8 total, 8 up, 8 in 2024-09-13T13:34:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:31 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:31 smithi043 ceph-mon[30106]: osdmap e281: 8 total, 8 up, 8 in 2024-09-13T13:34:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:31 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:31 smithi043 ceph-mon[40255]: osdmap e281: 8 total, 8 up, 8 in 2024-09-13T13:34:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:31 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:31 smithi189 ceph-mon[32912]: osdmap e281: 8 total, 8 up, 8 in 2024-09-13T13:34:33.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: pgmap v370: 196 pgs: 196 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: osdmap e282: 8 total, 8 up, 8 in 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: pgmap v370: 196 pgs: 196 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: osdmap e282: 8 total, 8 up, 8 in 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:33.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: pgmap v370: 196 pgs: 196 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: osdmap e282: 8 total, 8 up, 8 in 2024-09-13T13:34:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1153289005' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"} : dispatch 2024-09-13T13:34:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: osdmap e283: 8 total, 8 up, 8 in 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi043-76627-6","var": "pg_num","format": "json"} : dispatch 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:35.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-76332-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[40255]: pgmap v373: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: osdmap e283: 8 total, 8 up, 8 in 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi043-76627-6","var": "pg_num","format": "json"} : dispatch 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:35.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-76332-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:35.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:34 smithi043 ceph-mon[30106]: pgmap v373: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-76332-56"}]': finished 2024-09-13T13:34:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: osdmap e283: 8 total, 8 up, 8 in 2024-09-13T13:34:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi043-76627-6","var": "pg_num","format": "json"} : dispatch 2024-09-13T13:34:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-76332-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:34 smithi189 ceph-mon[32912]: pgmap v373: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:35.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:34:35.973 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:34:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-76332-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: osdmap e284: 8 total, 8 up, 8 in 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi043-76332-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:36.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-76332-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: osdmap e284: 8 total, 8 up, 8 in 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi043-76332-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:36.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:35 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-76332-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: osdmap e284: 8 total, 8 up, 8 in 2024-09-13T13:34:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-13T13:34:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi043-76332-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-13T13:34:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:36.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:35 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"}]': finished 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: osdmap e285: 8 total, 8 up, 8 in 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: pgmap v376: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-76332-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:37.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: osdmap e286: 8 total, 8 up, 8 in 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"}]': finished 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: osdmap e285: 8 total, 8 up, 8 in 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: pgmap v376: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-76332-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:34:37.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: osdmap e286: 8 total, 8 up, 8 in 2024-09-13T13:34:37.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-13T13:34:37.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-13T13:34:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_count","val": "8"}]': finished 2024-09-13T13:34:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: osdmap e285: 8 total, 8 up, 8 in 2024-09-13T13:34:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:34:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: pgmap v376: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-76332-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:34:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: osdmap e286: 8 total, 8 up, 8 in 2024-09-13T13:34:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-13T13:34:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 274:602f83fe:::foo:head 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9021 ms) 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 0 ls 1726234478,0 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 1 ls 1726234478,0 2024-09-13T13:34:38.202 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 2 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 3 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 4 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 5 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 6 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 7 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 8 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 9 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 10 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 11 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 12 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 13 ls 1726234478,0 2024-09-13T13:34:38.203 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 14 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 15 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 16 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 17 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 18 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 19 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 20 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 21 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 22 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 23 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 24 ls 1726234478,0 2024-09-13T13:34:38.204 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 25 ls 1726234478,0 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 26 ls 1726234478,0 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 27 ls 1726234478,0 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 28 ls 1726234478,0 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 29 ls 1726234478,0 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 30 ls 1726234478,0 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 31 ls 1726234478,0 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6cac518f:::0:head 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:02547ec2:::1:head 2024-09-13T13:34:38.205 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f905c69b:::2:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cfc208b3:::3:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d83876eb:::4:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b29083e3:::5:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c4fdafeb:::6:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5c6b0b28:::7:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bd63b0f1:::8:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e960b815:::9:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:52ea6a34:::10:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:89d3ae78:::11:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:de5d7c5f:::12:head 2024-09-13T13:34:38.206 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:566253c9:::13:head 2024-09-13T13:34:38.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:62a1935d:::14:head 2024-09-13T13:34:38.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:863748b0:::15:head 2024-09-13T13:34:38.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3958e169:::16:head 2024-09-13T13:34:38.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4d4dabf9:::17:head 2024-09-13T13:34:38.207 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:8391935d:::18:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:28883081:::19:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:69259c59:::20:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4bdb80b7:::21:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a11c5d71:::22:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:271af37b:::23:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:95b121be:::24:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:58d1031b:::25:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0a050783:::26:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c709704c:::27:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cbe56eaf:::28:head 2024-09-13T13:34:38.208 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:86b4b162:::29:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:70d89383:::30:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:dd450c7c:::31:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6d5729b1:::32:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c388f3fb:::33:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:56cfea31:::34:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9dbc1bf7:::35:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:40b74ccd:::36:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4d5aaf42:::37:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:920f362c:::38:head 2024-09-13T13:34:38.209 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6cc53222:::39:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9cad833f:::40:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1ea84d41:::41:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c4480ef6:::42:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a694361e:::43:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d1bd33e9:::44:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ddc2cd5d:::45:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:2b782207:::46:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7b187fca:::47:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:90ecdf6f:::48:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a5ed95fe:::49:head 2024-09-13T13:34:38.210 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ea0eaa55:::50:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f33ef17b:::51:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a0d1b2f6:::52:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:60c5229e:::53:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:edcbc575:::54:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:102cf253:::55:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:efb7fb0b:::56:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:50d0a326:::57:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d4dc5daf:::58:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3a130462:::59:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ec87ed71:::60:head 2024-09-13T13:34:38.211 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d5bc9454:::61:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3ddfe313:::62:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7c2816b9:::63:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:47e00e4d:::64:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c6410c18:::65:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b48ed237:::66:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cd63ad31:::67:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b179e92b:::68:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0d9f741a:::69:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6d3352ae:::70:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c6d5c19e:::71:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bc4729c3:::72:head 2024-09-13T13:34:38.212 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:77e930b9:::73:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0abeecfd:::74:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b7c37e15:::75:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b6378398:::76:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:02bd68de:::77:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cc795d2d:::78:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:630d4fea:::79:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e0d29ef5:::80:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:fd6f13d2:::81:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:606461d5:::82:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:eadbdc43:::83:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:8761d0bb:::84:head 2024-09-13T13:34:38.213 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9ef0186f:::85:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e0d41294:::86:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:961de695:::87:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1423148f:::88:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:633a8fa2:::89:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a8653809:::90:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3dac8b33:::91:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:35aad435:::92:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f6dcc343:::93:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:dbbdad87:::94:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1cb48ce0:::95:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:03cd461c:::96:head 2024-09-13T13:34:38.214 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:17a4ea99:::97:head 2024-09-13T13:34:38.215 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9993c9a7:::98:head 2024-09-13T13:34:38.215 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6394211c:::99:head 2024-09-13T13:34:38.215 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:94c7ae57:::100:head 2024-09-13T13:34:38.215 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6fdee5bb:::101:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9a477fd1:::102:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:eb850916:::103:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:affc56b9:::104:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b42dc814:::105:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f319f8f0:::106:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9a40b9de:::107:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:8b524f28:::108:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e3de589f:::109:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:90f90a5b:::110:head 2024-09-13T13:34:38.216 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a7b4f1d7:::111:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:af51766e:::112:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b6f90bd1:::113:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e0261208:::114:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c9569ef7:::115:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:61bebe50:::116:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:fe93412b:::117:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d3d38bee:::118:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3100ba0c:::119:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d0560ada:::120:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f0ea8b35:::121:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:766f231a:::122:head 2024-09-13T13:34:38.217 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a07a2582:::123:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bd7c6b3a:::124:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:fb2ddaff:::125:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4408e1fe:::126:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ee1df7a7:::127:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c3002909:::128:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4f48ffa9:::129:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:edf38733:::130:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c08425c0:::131:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5f902d98:::132:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:41ea2c93:::133:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:813cee13:::134:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0131818d:::135:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:26ba5a85:::136:head 2024-09-13T13:34:38.218 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:381b8a5a:::137:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:28797e47:::138:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bfca7f22:::139:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:36807075:::140:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:80b03975:::141:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5c15709b:::142:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f39ea15e:::143:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ea992956:::144:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:48887b1c:::145:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9f24a9dd:::146:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:987f100b:::147:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d2dd3581:::148:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7fed1808:::149:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c80b70e9:::150:head 2024-09-13T13:34:38.219 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:85ed90f9:::151:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:36428b24:::152:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d044c34a:::153:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7c18bf58:::154:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d1c21232:::155:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a7a3c575:::156:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:87da0633:::157:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d5ac3822:::158:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3f20522d:::159:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6ca26563:::160:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:532ce135:::161:head 2024-09-13T13:34:38.220 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c78863e6:::162:head 2024-09-13T13:34:39.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-13T13:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: osdmap e287: 8 total, 8 up, 8 in 2024-09-13T13:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi043-76627-43","var": "pg_num","format": "json"} : dispatch 2024-09-13T13:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: pgmap v379: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"} : dispatch 2024-09-13T13:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"} : dispatch 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: osdmap e287: 8 total, 8 up, 8 in 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi043-76627-43","var": "pg_num","format": "json"} : dispatch 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: pgmap v379: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"} : dispatch 2024-09-13T13:34:39.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"} : dispatch 2024-09-13T13:34:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-13T13:34:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: osdmap e287: 8 total, 8 up, 8 in 2024-09-13T13:34:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi043-76627-43","var": "pg_num","format": "json"} : dispatch 2024-09-13T13:34:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: pgmap v379: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:34:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"} : dispatch 2024-09-13T13:34:39.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"} : dispatch 2024-09-13T13:34:40.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"}]': finished 2024-09-13T13:34:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:39 smithi043 ceph-mon[30106]: osdmap e288: 8 total, 8 up, 8 in 2024-09-13T13:34:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"}]': finished 2024-09-13T13:34:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:39 smithi043 ceph-mon[40255]: osdmap e288: 8 total, 8 up, 8 in 2024-09-13T13:34:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-43"}]': finished 2024-09-13T13:34:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:39 smithi189 ceph-mon[32912]: osdmap e288: 8 total, 8 up, 8 in 2024-09-13T13:34:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[40255]: osdmap e289: 8 total, 8 up, 8 in 2024-09-13T13:34:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[40255]: pgmap v382: 196 pgs: 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[40255]: osdmap e290: 8 total, 8 up, 8 in 2024-09-13T13:34:41.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:41.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[30106]: osdmap e289: 8 total, 8 up, 8 in 2024-09-13T13:34:41.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:41.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[30106]: pgmap v382: 196 pgs: 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:41.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:41.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:40 smithi043 ceph-mon[30106]: osdmap e290: 8 total, 8 up, 8 in 2024-09-13T13:34:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:40 smithi189 ceph-mon[32912]: osdmap e289: 8 total, 8 up, 8 in 2024-09-13T13:34:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"} : dispatch 2024-09-13T13:34:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:40 smithi189 ceph-mon[32912]: pgmap v382: 196 pgs: 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3709524325' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-76332-57"}]': finished 2024-09-13T13:34:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:40 smithi189 ceph-mon[32912]: osdmap e290: 8 total, 8 up, 8 in 2024-09-13T13:34:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-76332-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-76332-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: osdmap e291: 8 total, 8 up, 8 in 2024-09-13T13:34:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-76332-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:42.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-76332-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-76332-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: osdmap e291: 8 total, 8 up, 8 in 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-76332-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-76332-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:42.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:42.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-76332-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:42.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: osdmap e291: 8 total, 8 up, 8 in 2024-09-13T13:34:42.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-76332-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:42.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:42.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:34:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:43 smithi189 ceph-mon[32912]: pgmap v385: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:34:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:43 smithi189 ceph-mon[32912]: osdmap e292: 8 total, 8 up, 8 in 2024-09-13T13:34:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:34:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:42 smithi043 ceph-mon[30106]: pgmap v385: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:34:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:42 smithi043 ceph-mon[30106]: osdmap e292: 8 total, 8 up, 8 in 2024-09-13T13:34:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:34:43.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:42 smithi043 ceph-mon[40255]: pgmap v385: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:43.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:34:43.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:34:43.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:43 smithi043 ceph-mon[40255]: osdmap e292: 8 total, 8 up, 8 in 2024-09-13T13:34:43.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-76332-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"}]': finished 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[30106]: osdmap e293: 8 total, 8 up, 8 in 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[30106]: pgmap v388: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-76332-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"}]': finished 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[40255]: osdmap e293: 8 total, 8 up, 8 in 2024-09-13T13:34:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:34:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:34:45.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:44 smithi043 ceph-mon[40255]: pgmap v388: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-76332-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_count","val": "3"}]': finished 2024-09-13T13:34:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:44 smithi189 ceph-mon[32912]: osdmap e293: 8 total, 8 up, 8 in 2024-09-13T13:34:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:34:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:34:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:44 smithi189 ceph-mon[32912]: pgmap v388: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:45.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:34:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:34:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:34:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:34:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:45 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"}]': finished 2024-09-13T13:34:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:45 smithi189 ceph-mon[32912]: osdmap e294: 8 total, 8 up, 8 in 2024-09-13T13:34:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:34:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:34:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:45 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"}]': finished 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[30106]: osdmap e294: 8 total, 8 up, 8 in 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_period","val": "3"}]': finished 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[40255]: osdmap e294: 8 total, 8 up, 8 in 2024-09-13T13:34:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:34:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:34:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:45 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:34:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: osdmap e295: 8 total, 8 up, 8 in 2024-09-13T13:34:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:34:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: pgmap v391: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: osdmap e296: 8 total, 8 up, 8 in 2024-09-13T13:34:47.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: osdmap e295: 8 total, 8 up, 8 in 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: pgmap v391: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:34:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:47.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-13T13:34:47.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: osdmap e296: 8 total, 8 up, 8 in 2024-09-13T13:34:47.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:47.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:34:47.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: osdmap e295: 8 total, 8 up, 8 in 2024-09-13T13:34:47.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:34:47.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:47.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:34:47.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: pgmap v391: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:47.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:34:47.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:47.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:47.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-13T13:34:47.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: osdmap e296: 8 total, 8 up, 8 in 2024-09-13T13:34:47.496 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"} : dispatch 2024-09-13T13:34:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:48 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:34:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:48 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:34:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:48 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:48 smithi189 ceph-mon[32912]: osdmap e297: 8 total, 8 up, 8 in 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[30106]: osdmap e297: 8 total, 8 up, 8 in 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:34:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1225510556' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-76332-58"}]': finished 2024-09-13T13:34:48.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:48 smithi043 ceph-mon[40255]: osdmap e297: 8 total, 8 up, 8 in 2024-09-13T13:34:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:49 smithi189 ceph-mon[32912]: pgmap v394: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[30106]: pgmap v394: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:49 smithi043 ceph-mon[40255]: pgmap v394: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:34:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:50 smithi189 ceph-mon[32912]: osdmap e298: 8 total, 8 up, 8 in 2024-09-13T13:34:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-76332-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[40255]: osdmap e298: 8 total, 8 up, 8 in 2024-09-13T13:34:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-76332-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:34:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[30106]: osdmap e298: 8 total, 8 up, 8 in 2024-09-13T13:34:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-76332-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:50.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:34:51.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:51 smithi189 ceph-mon[32912]: osdmap e299: 8 total, 8 up, 8 in 2024-09-13T13:34:51.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:51 smithi189 ceph-mon[32912]: pgmap v397: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:51.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-76332-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:51.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:51 smithi189 ceph-mon[32912]: osdmap e300: 8 total, 8 up, 8 in 2024-09-13T13:34:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[40255]: osdmap e299: 8 total, 8 up, 8 in 2024-09-13T13:34:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[40255]: pgmap v397: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-76332-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[40255]: osdmap e300: 8 total, 8 up, 8 in 2024-09-13T13:34:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[30106]: osdmap e299: 8 total, 8 up, 8 in 2024-09-13T13:34:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[30106]: pgmap v397: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-76332-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:51 smithi043 ceph-mon[30106]: osdmap e300: 8 total, 8 up, 8 in 2024-09-13T13:34:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:52 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:52 smithi189 ceph-mon[32912]: osdmap e301: 8 total, 8 up, 8 in 2024-09-13T13:34:52.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:52 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:52.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:52 smithi043 ceph-mon[30106]: osdmap e301: 8 total, 8 up, 8 in 2024-09-13T13:34:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:52 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:52 smithi043 ceph-mon[40255]: osdmap e301: 8 total, 8 up, 8 in 2024-09-13T13:34:53.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:53 smithi189 ceph-mon[32912]: pgmap v400: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:53.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:53 smithi189 ceph-mon[32912]: osdmap e302: 8 total, 8 up, 8 in 2024-09-13T13:34:53.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:53 smithi043 ceph-mon[30106]: pgmap v400: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:53 smithi043 ceph-mon[30106]: osdmap e302: 8 total, 8 up, 8 in 2024-09-13T13:34:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:53.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:53 smithi043 ceph-mon[40255]: pgmap v400: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:34:53.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:53 smithi043 ceph-mon[40255]: osdmap e302: 8 total, 8 up, 8 in 2024-09-13T13:34:53.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:55 smithi189 ceph-mon[32912]: osdmap e303: 8 total, 8 up, 8 in 2024-09-13T13:34:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:55 smithi189 ceph-mon[32912]: pgmap v403: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[30106]: osdmap e303: 8 total, 8 up, 8 in 2024-09-13T13:34:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[30106]: pgmap v403: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[40255]: osdmap e303: 8 total, 8 up, 8 in 2024-09-13T13:34:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"} : dispatch 2024-09-13T13:34:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-mon[40255]: pgmap v403: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:34:55.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:34:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:34:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:34:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:34:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:34:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:56 smithi189 ceph-mon[32912]: osdmap e304: 8 total, 8 up, 8 in 2024-09-13T13:34:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:56.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:56 smithi043 ceph-mon[30106]: osdmap e304: 8 total, 8 up, 8 in 2024-09-13T13:34:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1856479178' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-76332-59"}]': finished 2024-09-13T13:34:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:56 smithi043 ceph-mon[40255]: osdmap e304: 8 total, 8 up, 8 in 2024-09-13T13:34:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:34:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:57 smithi189 ceph-mon[32912]: osdmap e305: 8 total, 8 up, 8 in 2024-09-13T13:34:57.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3891248191' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:57.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:57 smithi189 ceph-mon[32912]: pgmap v406: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:34:57.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:57 smithi043 ceph-mon[30106]: osdmap e305: 8 total, 8 up, 8 in 2024-09-13T13:34:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3891248191' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:57 smithi043 ceph-mon[30106]: pgmap v406: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:34:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:57 smithi043 ceph-mon[40255]: osdmap e305: 8 total, 8 up, 8 in 2024-09-13T13:34:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3891248191' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:34:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:57 smithi043 ceph-mon[40255]: pgmap v406: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:34:58.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7462ddf6:::16.RoundTripAppendPP (2965 ms) 2024-09-13T13:34:58.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3012 ms) 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3005 ms) 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3011 ms) 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: setting pool EIO 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: max_success 99, min_failed 100 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4008 ms) 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3002 ms) 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (111342 ms total) 2024-09-13T13:34:58.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3015 ms) 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9027 ms) 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14942 ms) 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3005 ms) 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29989 ms total) 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-13T13:34:58.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3011 ms) 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3011 ms total) 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13988 ms) 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6994 ms) 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7019 ms) 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7023 ms) 2024-09-13T13:34:58.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3009 ms) 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7024 ms) 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6932 ms) 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7025 ms) 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7027 ms) 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7018 ms) 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-13T13:34:58.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7020 ms) 2024-09-13T13:34:58.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-13T13:34:58.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7010 ms) 2024-09-13T13:34:58.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-13T13:34:58.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7034 ms) 2024-09-13T13:34:58.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-09-13T13:34:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:58 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3891248191' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:58 smithi189 ceph-mon[32912]: osdmap e306: 8 total, 8 up, 8 in 2024-09-13T13:34:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:58 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3891248191' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:58 smithi043 ceph-mon[30106]: osdmap e306: 8 total, 8 up, 8 in 2024-09-13T13:34:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:58 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:34:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3891248191' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-76332-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:34:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:58 smithi043 ceph-mon[40255]: osdmap e306: 8 total, 8 up, 8 in 2024-09-13T13:34:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: osdmap e307: 8 total, 8 up, 8 in 2024-09-13T13:34:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:34:59 smithi189 ceph-mon[32912]: pgmap v409: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:34:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: osdmap e307: 8 total, 8 up, 8 in 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[30106]: pgmap v409: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:34:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: osdmap e307: 8 total, 8 up, 8 in 2024-09-13T13:34:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:34:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:34:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:34:59 smithi043 ceph-mon[40255]: pgmap v409: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:35:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: osdmap e308: 8 total, 8 up, 8 in 2024-09-13T13:35:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"} : dispatch 2024-09-13T13:35:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"} : dispatch 2024-09-13T13:35:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: osdmap e308: 8 total, 8 up, 8 in 2024-09-13T13:35:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"} : dispatch 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"} : dispatch 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-76332-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: osdmap e308: 8 total, 8 up, 8 in 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"} : dispatch 2024-09-13T13:35:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"} : dispatch 2024-09-13T13:35:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"}]': finished 2024-09-13T13:35:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:01 smithi189 ceph-mon[32912]: osdmap e309: 8 total, 8 up, 8 in 2024-09-13T13:35:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:01 smithi189 ceph-mon[32912]: pgmap v412: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:35:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"}]': finished 2024-09-13T13:35:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:01 smithi043 ceph-mon[30106]: osdmap e309: 8 total, 8 up, 8 in 2024-09-13T13:35:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:01 smithi043 ceph-mon[30106]: pgmap v412: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:35:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-45"}]': finished 2024-09-13T13:35:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:01 smithi043 ceph-mon[40255]: osdmap e309: 8 total, 8 up, 8 in 2024-09-13T13:35:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:01 smithi043 ceph-mon[40255]: pgmap v412: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-13T13:35:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:02 smithi189 ceph-mon[32912]: osdmap e310: 8 total, 8 up, 8 in 2024-09-13T13:35:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:02 smithi043 ceph-mon[30106]: osdmap e310: 8 total, 8 up, 8 in 2024-09-13T13:35:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-76332-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:02 smithi043 ceph-mon[40255]: osdmap e310: 8 total, 8 up, 8 in 2024-09-13T13:35:03.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: osdmap e311: 8 total, 8 up, 8 in 2024-09-13T13:35:03.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:03.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:03.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: pgmap v415: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:03.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: osdmap e312: 8 total, 8 up, 8 in 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: osdmap e311: 8 total, 8 up, 8 in 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: pgmap v415: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: osdmap e312: 8 total, 8 up, 8 in 2024-09-13T13:35:03.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:03.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: osdmap e311: 8 total, 8 up, 8 in 2024-09-13T13:35:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: pgmap v415: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: osdmap e312: 8 total, 8 up, 8 in 2024-09-13T13:35:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:03.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: osdmap e313: 8 total, 8 up, 8 in 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:04.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: osdmap e313: 8 total, 8 up, 8 in 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: osdmap e313: 8 total, 8 up, 8 in 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/896844388' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"} : dispatch 2024-09-13T13:35:04.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-mon[32912]: pgmap v418: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"}]': finished 2024-09-13T13:35:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-mon[32912]: osdmap e314: 8 total, 8 up, 8 in 2024-09-13T13:35:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"} : dispatch 2024-09-13T13:35:05.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"} : dispatch 2024-09-13T13:35:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[30106]: pgmap v418: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"}]': finished 2024-09-13T13:35:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[30106]: osdmap e314: 8 total, 8 up, 8 in 2024-09-13T13:35:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"} : dispatch 2024-09-13T13:35:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"} : dispatch 2024-09-13T13:35:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[40255]: pgmap v418: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-76332-61"}]': finished 2024-09-13T13:35:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-47"}]': finished 2024-09-13T13:35:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[40255]: osdmap e314: 8 total, 8 up, 8 in 2024-09-13T13:35:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"} : dispatch 2024-09-13T13:35:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"} : dispatch 2024-09-13T13:35:05.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:35:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:35:05.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:35:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:35:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:35:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:06.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"}]': finished 2024-09-13T13:35:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: osdmap e315: 8 total, 8 up, 8 in 2024-09-13T13:35:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-76332-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"}]': finished 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: osdmap e315: 8 total, 8 up, 8 in 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-76332-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-47", "mode": "writeback"}]': finished 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: osdmap e315: 8 total, 8 up, 8 in 2024-09-13T13:35:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-76332-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:06.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:06.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:07 smithi189 ceph-mon[32912]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:35:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:35:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:07 smithi189 ceph-mon[32912]: osdmap e316: 8 total, 8 up, 8 in 2024-09-13T13:35:07.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[30106]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[30106]: osdmap e316: 8 total, 8 up, 8 in 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[40255]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[40255]: osdmap e316: 8 total, 8 up, 8 in 2024-09-13T13:35:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-76332-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:35:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:09 smithi189 ceph-mon[32912]: osdmap e317: 8 total, 8 up, 8 in 2024-09-13T13:35:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:09.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:09 smithi189 ceph-mon[32912]: pgmap v424: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:35:09.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-76332-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:35:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[40255]: osdmap e317: 8 total, 8 up, 8 in 2024-09-13T13:35:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[40255]: pgmap v424: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:35:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-76332-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:35:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[30106]: osdmap e317: 8 total, 8 up, 8 in 2024-09-13T13:35:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:09 smithi043 ceph-mon[30106]: pgmap v424: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:35:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:10 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:10 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:35:10.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:10 smithi189 ceph-mon[32912]: osdmap e318: 8 total, 8 up, 8 in 2024-09-13T13:35:10.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:10.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[40255]: osdmap e318: 8 total, 8 up, 8 in 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:35:10.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[30106]: osdmap e318: 8 total, 8 up, 8 in 2024-09-13T13:35:10.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:10.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:35:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:11 smithi189 ceph-mon[32912]: osdmap e319: 8 total, 8 up, 8 in 2024-09-13T13:35:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:35:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:35:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:11 smithi189 ceph-mon[32912]: pgmap v427: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-13T13:35:11.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:35:11.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[30106]: osdmap e319: 8 total, 8 up, 8 in 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[30106]: pgmap v427: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[40255]: osdmap e319: 8 total, 8 up, 8 in 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:35:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:35:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[40255]: pgmap v427: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-13T13:35:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: osdmap e320: 8 total, 8 up, 8 in 2024-09-13T13:35:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:35:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:35:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-13T13:35:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:12 smithi189 ceph-mon[32912]: osdmap e321: 8 total, 8 up, 8 in 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: osdmap e320: 8 total, 8 up, 8 in 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-13T13:35:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[30106]: osdmap e321: 8 total, 8 up, 8 in 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: osdmap e320: 8 total, 8 up, 8 in 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"} : dispatch 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2212884735' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-76332-62"}]': finished 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-13T13:35:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:12 smithi043 ceph-mon[40255]: osdmap e321: 8 total, 8 up, 8 in 2024-09-13T13:35:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-76332-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:13 smithi189 ceph-mon[32912]: pgmap v430: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-76332-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[30106]: pgmap v430: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-76332-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:13 smithi043 ceph-mon[40255]: pgmap v430: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-76332-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:14 smithi189 ceph-mon[32912]: osdmap e322: 8 total, 8 up, 8 in 2024-09-13T13:35:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-76332-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:14.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-76332-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[30106]: osdmap e322: 8 total, 8 up, 8 in 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-76332-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-76332-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[40255]: osdmap e322: 8 total, 8 up, 8 in 2024-09-13T13:35:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-76332-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:14.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"}]': finished 2024-09-13T13:35:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-mon[32912]: osdmap e323: 8 total, 8 up, 8 in 2024-09-13T13:35:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-mon[32912]: pgmap v433: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"}]': finished 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[30106]: osdmap e323: 8 total, 8 up, 8 in 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[30106]: pgmap v433: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"}]': finished 2024-09-13T13:35:15.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[40255]: osdmap e323: 8 total, 8 up, 8 in 2024-09-13T13:35:15.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:15.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:15.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:15.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-47"} : dispatch 2024-09-13T13:35:15.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-mon[40255]: pgmap v433: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:15.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:35:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:35:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:35:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:35:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:35:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-76332-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:16 smithi189 ceph-mon[32912]: osdmap e324: 8 total, 8 up, 8 in 2024-09-13T13:35:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:16.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-76332-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:16 smithi043 ceph-mon[30106]: osdmap e324: 8 total, 8 up, 8 in 2024-09-13T13:35:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-76332-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:16 smithi043 ceph-mon[40255]: osdmap e324: 8 total, 8 up, 8 in 2024-09-13T13:35:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: osdmap e325: 8 total, 8 up, 8 in 2024-09-13T13:35:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: pgmap v436: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:35:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: osdmap e326: 8 total, 8 up, 8 in 2024-09-13T13:35:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: osdmap e325: 8 total, 8 up, 8 in 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: pgmap v436: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: osdmap e326: 8 total, 8 up, 8 in 2024-09-13T13:35:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: osdmap e325: 8 total, 8 up, 8 in 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: pgmap v436: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: osdmap e326: 8 total, 8 up, 8 in 2024-09-13T13:35:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: osdmap e327: 8 total, 8 up, 8 in 2024-09-13T13:35:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: osdmap e327: 8 total, 8 up, 8 in 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:18.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: osdmap e327: 8 total, 8 up, 8 in 2024-09-13T13:35:18.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:18.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"} : dispatch 2024-09-13T13:35:18.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:19.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[30106]: pgmap v439: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"}]': finished 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[30106]: osdmap e328: 8 total, 8 up, 8 in 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"} : dispatch 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"} : dispatch 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[40255]: pgmap v439: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:19.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"}]': finished 2024-09-13T13:35:19.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[40255]: osdmap e328: 8 total, 8 up, 8 in 2024-09-13T13:35:19.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"} : dispatch 2024-09-13T13:35:19.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"} : dispatch 2024-09-13T13:35:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:19 smithi189 ceph-mon[32912]: pgmap v439: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:35:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/642855351' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-76332-63"}]': finished 2024-09-13T13:35:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-49"}]': finished 2024-09-13T13:35:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:19 smithi189 ceph-mon[32912]: osdmap e328: 8 total, 8 up, 8 in 2024-09-13T13:35:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"} : dispatch 2024-09-13T13:35:19.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"}]': finished 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: osdmap e329: 8 total, 8 up, 8 in 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"}]': finished 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: osdmap e329: 8 total, 8 up, 8 in 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-49", "mode": "readproxy"}]': finished 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-76332-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: osdmap e329: 8 total, 8 up, 8 in 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:20.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:21 smithi043 ceph-mon[30106]: pgmap v442: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:35:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:21 smithi043 ceph-mon[30106]: osdmap e330: 8 total, 8 up, 8 in 2024-09-13T13:35:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:21 smithi043 ceph-mon[40255]: pgmap v442: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:35:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:21 smithi043 ceph-mon[40255]: osdmap e330: 8 total, 8 up, 8 in 2024-09-13T13:35:21.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:21 smithi189 ceph-mon[32912]: pgmap v442: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:35:21.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:21 smithi189 ceph-mon[32912]: osdmap e330: 8 total, 8 up, 8 in 2024-09-13T13:35:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:23 smithi189 ceph-mon[32912]: osdmap e331: 8 total, 8 up, 8 in 2024-09-13T13:35:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:23 smithi189 ceph-mon[32912]: pgmap v445: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:35:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:23 smithi043 ceph-mon[30106]: osdmap e331: 8 total, 8 up, 8 in 2024-09-13T13:35:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:23 smithi043 ceph-mon[30106]: pgmap v445: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:35:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-76332-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:23 smithi043 ceph-mon[40255]: osdmap e331: 8 total, 8 up, 8 in 2024-09-13T13:35:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:23 smithi043 ceph-mon[40255]: pgmap v445: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:35:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:24 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:24 smithi189 ceph-mon[32912]: osdmap e332: 8 total, 8 up, 8 in 2024-09-13T13:35:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:24 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:24 smithi043 ceph-mon[30106]: osdmap e332: 8 total, 8 up, 8 in 2024-09-13T13:35:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:24 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:24 smithi043 ceph-mon[40255]: osdmap e332: 8 total, 8 up, 8 in 2024-09-13T13:35:25.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:25 smithi189 ceph-mon[32912]: osdmap e333: 8 total, 8 up, 8 in 2024-09-13T13:35:25.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:25.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:25.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:25 smithi189 ceph-mon[32912]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[30106]: osdmap e333: 8 total, 8 up, 8 in 2024-09-13T13:35:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[30106]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[40255]: osdmap e333: 8 total, 8 up, 8 in 2024-09-13T13:35:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-mon[40255]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:25.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:35:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:35:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:35:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:35:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:35:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[30106]: osdmap e334: 8 total, 8 up, 8 in 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[40255]: osdmap e334: 8 total, 8 up, 8 in 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:26.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:26 smithi189 ceph-mon[32912]: osdmap e334: 8 total, 8 up, 8 in 2024-09-13T13:35:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3566815831' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"} : dispatch 2024-09-13T13:35:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: osdmap e335: 8 total, 8 up, 8 in 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[30106]: pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: osdmap e335: 8 total, 8 up, 8 in 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:27 smithi043 ceph-mon[40255]: pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-76332-64"}]': finished 2024-09-13T13:35:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: osdmap e335: 8 total, 8 up, 8 in 2024-09-13T13:35:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:27.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:27.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:27.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:27 smithi189 ceph-mon[32912]: pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[30106]: osdmap e336: 8 total, 8 up, 8 in 2024-09-13T13:35:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:28.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[40255]: osdmap e336: 8 total, 8 up, 8 in 2024-09-13T13:35:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:28.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-76332-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:28 smithi189 ceph-mon[32912]: osdmap e336: 8 total, 8 up, 8 in 2024-09-13T13:35:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[30106]: osdmap e337: 8 total, 8 up, 8 in 2024-09-13T13:35:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[30106]: pgmap v454: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[30106]: osdmap e338: 8 total, 8 up, 8 in 2024-09-13T13:35:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[40255]: osdmap e337: 8 total, 8 up, 8 in 2024-09-13T13:35:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[40255]: pgmap v454: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:29 smithi043 ceph-mon[40255]: osdmap e338: 8 total, 8 up, 8 in 2024-09-13T13:35:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:29 smithi189 ceph-mon[32912]: osdmap e337: 8 total, 8 up, 8 in 2024-09-13T13:35:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:29 smithi189 ceph-mon[32912]: pgmap v454: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-76332-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:29 smithi189 ceph-mon[32912]: osdmap e338: 8 total, 8 up, 8 in 2024-09-13T13:35:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[30106]: osdmap e339: 8 total, 8 up, 8 in 2024-09-13T13:35:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[30106]: pgmap v457: 236 pgs: 4 creating+activating, 232 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[40255]: osdmap e339: 8 total, 8 up, 8 in 2024-09-13T13:35:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[40255]: pgmap v457: 236 pgs: 4 creating+activating, 232 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:31 smithi189 ceph-mon[32912]: osdmap e339: 8 total, 8 up, 8 in 2024-09-13T13:35:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:31 smithi189 ceph-mon[32912]: pgmap v457: 236 pgs: 4 creating+activating, 232 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:32.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[30106]: osdmap e340: 8 total, 8 up, 8 in 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[40255]: osdmap e340: 8 total, 8 up, 8 in 2024-09-13T13:35:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:32.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:32.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:32 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:32.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:32.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:32 smithi189 ceph-mon[32912]: osdmap e340: 8 total, 8 up, 8 in 2024-09-13T13:35:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:32.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:33.199 INFO:tasks.workunit.client.0.smithi043.stdout:3:head 2024-09-13T13:35:33.233 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5d165639:::164:head 2024-09-13T13:35:33.233 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f43765fc:::165:head 2024-09-13T13:35:33.233 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b4c720e9:::166:head 2024-09-13T13:35:33.233 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e694b040:::167:head 2024-09-13T13:35:33.233 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:afa38db2:::168:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:77ba9f53:::169:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:87495034:::170:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7c96bf0e:::171:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:dbe346cc:::172:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e943ec24:::173:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f97a9c0c:::174:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6f26e74d:::175:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4f95e106:::176:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0e6f2f8f:::177:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:05db05f1:::178:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:38a78d66:::179:head 2024-09-13T13:35:33.234 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d095610b:::180:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a1a9d709:::181:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1e5d39db:::182:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f7df4fb9:::183:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:03a7f161:::184:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ba70721e:::185:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:28e5662d:::186:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:973d52de:::187:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4303eb1c:::188:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b990b48e:::189:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:29b8165b:::190:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3547f197:::191:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7e260936:::192:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1abec7b1:::193:head 2024-09-13T13:35:33.235 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:10fdda93:::194:head 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:15817eea:::195:head 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:770bab57:::196:head 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ed9e13e7:::197:head 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:71471a8f:::198:head 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:10fb1d02:::199:head 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8025 ms) 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,0 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1726234487 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,0 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,0 2024-09-13T13:35:33.236 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,1726234493,1726234494,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,1726234493,1726234494,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,1726234493,1726234494,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,1726234493,1726234494,1726234496,1726234497,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,1726234493,1726234494,1726234496,1726234497,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234487,1726234490,1726234491,1726234493,1726234494,1726234496,1726234497,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726234491,1726234493,1726234494,1726234496,1726234497,1726234499,1726234500,0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1726234491, trimmed 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21089 ms) 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-09-13T13:35:33.237 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-13T13:35:33.238 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14057 ms) 2024-09-13T13:35:33.238 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"}]': finished 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: osdmap e341: 8 total, 8 up, 8 in 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[30106]: pgmap v460: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"}]': finished 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: osdmap e341: 8 total, 8 up, 8 in 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:33.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:33 smithi043 ceph-mon[40255]: pgmap v460: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"}]': finished 2024-09-13T13:35:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: osdmap e341: 8 total, 8 up, 8 in 2024-09-13T13:35:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1566289458' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"} : dispatch 2024-09-13T13:35:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-49"} : dispatch 2024-09-13T13:35:33.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:33 smithi189 ceph-mon[32912]: pgmap v460: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:34.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: osdmap e342: 8 total, 8 up, 8 in 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: osdmap e342: 8 total, 8 up, 8 in 2024-09-13T13:35:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:34.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-76332-65"}]': finished 2024-09-13T13:35:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: osdmap e342: 8 total, 8 up, 8 in 2024-09-13T13:35:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:34.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:35:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: osdmap e343: 8 total, 8 up, 8 in 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: pgmap v463: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: osdmap e343: 8 total, 8 up, 8 in 2024-09-13T13:35:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: pgmap v463: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-76332-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: osdmap e343: 8 total, 8 up, 8 in 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: pgmap v463: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:35.498 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:35.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:35:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:35:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:35:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:35:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:35:36.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[40255]: osdmap e344: 8 total, 8 up, 8 in 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[30106]: osdmap e344: 8 total, 8 up, 8 in 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:36 smithi189 ceph-mon[32912]: osdmap e344: 8 total, 8 up, 8 in 2024-09-13T13:35:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:35:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[30106]: osdmap e345: 8 total, 8 up, 8 in 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[30106]: pgmap v466: 236 pgs: 8 unknown, 17 creating+activating, 15 creating+peering, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[40255]: osdmap e345: 8 total, 8 up, 8 in 2024-09-13T13:35:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:37.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:37.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:37 smithi043 ceph-mon[40255]: pgmap v466: 236 pgs: 8 unknown, 17 creating+activating, 15 creating+peering, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-76332-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:35:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:37 smithi189 ceph-mon[32912]: osdmap e345: 8 total, 8 up, 8 in 2024-09-13T13:35:37.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:37.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:37.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:37 smithi189 ceph-mon[32912]: pgmap v466: 236 pgs: 8 unknown, 17 creating+activating, 15 creating+peering, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"}]': finished 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: osdmap e346: 8 total, 8 up, 8 in 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"} : dispatch 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"} : dispatch 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"}]': finished 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: osdmap e347: 8 total, 8 up, 8 in 2024-09-13T13:35:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"}]': finished 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: osdmap e346: 8 total, 8 up, 8 in 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"} : dispatch 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"} : dispatch 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"}]': finished 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: osdmap e347: 8 total, 8 up, 8 in 2024-09-13T13:35:38.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:38.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-51"}]': finished 2024-09-13T13:35:38.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: osdmap e346: 8 total, 8 up, 8 in 2024-09-13T13:35:38.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"} : dispatch 2024-09-13T13:35:38.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"} : dispatch 2024-09-13T13:35:38.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:35:38.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-51", "mode": "writeback"}]': finished 2024-09-13T13:35:38.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: osdmap e347: 8 total, 8 up, 8 in 2024-09-13T13:35:38.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: pgmap v469: 228 pgs: 17 creating+activating, 15 creating+peering, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: osdmap e348: 8 total, 8 up, 8 in 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: pgmap v469: 228 pgs: 17 creating+activating, 15 creating+peering, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: osdmap e348: 8 total, 8 up, 8 in 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:39.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:39.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:39.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:35:39.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: pgmap v469: 228 pgs: 17 creating+activating, 15 creating+peering, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:35:39.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:39.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:35:39.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: osdmap e348: 8 total, 8 up, 8 in 2024-09-13T13:35:39.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2274156103' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:39.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"} : dispatch 2024-09-13T13:35:39.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3017 ms) 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7029 ms) 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7018 ms) 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7040 ms) 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7030 ms) 2024-09-13T13:35:40.231 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7039 ms) 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7025 ms) 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139322 ms total) 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (283665 ms total) 2024-09-13T13:35:40.232 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[30106]: osdmap e349: 8 total, 8 up, 8 in 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:40.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:35:40.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[40255]: osdmap e349: 8 total, 8 up, 8 in 2024-09-13T13:35:40.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:40.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:40.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:40 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:35:40.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-76332-66"}]': finished 2024-09-13T13:35:40.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:35:40.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:40 smithi189 ceph-mon[32912]: osdmap e349: 8 total, 8 up, 8 in 2024-09-13T13:35:40.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:40.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:35:41.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[30106]: pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[30106]: osdmap e350: 8 total, 8 up, 8 in 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[40255]: pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:35:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[40255]: osdmap e350: 8 total, 8 up, 8 in 2024-09-13T13:35:41.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:41.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:41.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:41 smithi189 ceph-mon[32912]: pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-13T13:35:41.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:41 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:35:41.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:35:41.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:41 smithi189 ceph-mon[32912]: osdmap e350: 8 total, 8 up, 8 in 2024-09-13T13:35:41.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:41.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:35:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[30106]: osdmap e351: 8 total, 8 up, 8 in 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[30106]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[40255]: osdmap e351: 8 total, 8 up, 8 in 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:35:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:43 smithi043 ceph-mon[40255]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-13T13:35:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:35:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:43 smithi189 ceph-mon[32912]: osdmap e351: 8 total, 8 up, 8 in 2024-09-13T13:35:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:35:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:35:43.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:43 smithi189 ceph-mon[32912]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-13T13:35:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"}]': finished 2024-09-13T13:35:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:44 smithi043 ceph-mon[30106]: osdmap e352: 8 total, 8 up, 8 in 2024-09-13T13:35:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"}]': finished 2024-09-13T13:35:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:44 smithi043 ceph-mon[40255]: osdmap e352: 8 total, 8 up, 8 in 2024-09-13T13:35:44.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-51","var": "target_max_objects","val": "1"}]': finished 2024-09-13T13:35:44.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:44 smithi189 ceph-mon[32912]: osdmap e352: 8 total, 8 up, 8 in 2024-09-13T13:35:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:45 smithi043 ceph-mon[30106]: pgmap v477: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:45 smithi043 ceph-mon[40255]: pgmap v477: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:45.509 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:45 smithi189 ceph-mon[32912]: pgmap v477: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:35:45.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:35:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:35:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:35:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:35:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:35:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:46 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-13T13:35:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:46 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-13T13:35:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:46 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-13T13:35:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:47 smithi189 ceph-mon[32912]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:47 smithi043 ceph-mon[30106]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:47 smithi043 ceph-mon[40255]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:48 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:35:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:48 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:35:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:48 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:35:48.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:48 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:35:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:35:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:35:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:35:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:35:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:35:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:35:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:35:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:48 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:35:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:49 smithi189 ceph-mon[32912]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 0 op/s 2024-09-13T13:35:49.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:49 smithi043 ceph-mon[30106]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 0 op/s 2024-09-13T13:35:49.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:49 smithi043 ceph-mon[40255]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 0 op/s 2024-09-13T13:35:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:35:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:51 smithi189 ceph-mon[32912]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:35:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:51 smithi043 ceph-mon[30106]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:35:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:35:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:51 smithi043 ceph-mon[40255]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:53 smithi189 ceph-mon[32912]: pgmap v481: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:53 smithi043 ceph-mon[30106]: pgmap v481: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:53 smithi043 ceph-mon[40255]: pgmap v481: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:54 smithi189 ceph-mon[32912]: osdmap e353: 8 total, 8 up, 8 in 2024-09-13T13:35:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"}]': finished 2024-09-13T13:35:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:54 smithi189 ceph-mon[32912]: osdmap e354: 8 total, 8 up, 8 in 2024-09-13T13:35:54.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[30106]: osdmap e353: 8 total, 8 up, 8 in 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"}]': finished 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[30106]: osdmap e354: 8 total, 8 up, 8 in 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[40255]: osdmap e353: 8 total, 8 up, 8 in 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"}]': finished 2024-09-13T13:35:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:54 smithi043 ceph-mon[40255]: osdmap e354: 8 total, 8 up, 8 in 2024-09-13T13:35:55.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:35:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:35:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:55 smithi189 ceph-mon[32912]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:55.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:55.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:55.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:55.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[30106]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[40255]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:55.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:55.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-51"} : dispatch 2024-09-13T13:35:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:35:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:35:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:35:56.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:56 smithi189 ceph-mon[32912]: osdmap e355: 8 total, 8 up, 8 in 2024-09-13T13:35:56.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:56 smithi043 ceph-mon[30106]: osdmap e355: 8 total, 8 up, 8 in 2024-09-13T13:35:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:56 smithi043 ceph-mon[40255]: osdmap e355: 8 total, 8 up, 8 in 2024-09-13T13:35:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:35:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:57 smithi189 ceph-mon[32912]: pgmap v486: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-13T13:35:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:57 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-13T13:35:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:57 smithi189 ceph-mon[32912]: osdmap e356: 8 total, 8 up, 8 in 2024-09-13T13:35:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:57.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[30106]: pgmap v486: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-13T13:35:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-13T13:35:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[30106]: osdmap e356: 8 total, 8 up, 8 in 2024-09-13T13:35:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[40255]: pgmap v486: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-13T13:35:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-13T13:35:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[40255]: osdmap e356: 8 total, 8 up, 8 in 2024-09-13T13:35:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:57.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:35:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:58 smithi189 ceph-mon[32912]: osdmap e357: 8 total, 8 up, 8 in 2024-09-13T13:35:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-53"} : dispatch 2024-09-13T13:35:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-53"} : dispatch 2024-09-13T13:35:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[30106]: osdmap e357: 8 total, 8 up, 8 in 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-53"} : dispatch 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-53"} : dispatch 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[40255]: osdmap e357: 8 total, 8 up, 8 in 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-53"} : dispatch 2024-09-13T13:35:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-53"} : dispatch 2024-09-13T13:35:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:59 smithi189 ceph-mon[32912]: pgmap v489: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-13T13:35:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:35:59 smithi189 ceph-mon[32912]: osdmap e358: 8 total, 8 up, 8 in 2024-09-13T13:35:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:59 smithi043 ceph-mon[40255]: pgmap v489: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-13T13:35:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:35:59 smithi043 ceph-mon[40255]: osdmap e358: 8 total, 8 up, 8 in 2024-09-13T13:35:59.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:59 smithi043 ceph-mon[30106]: pgmap v489: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-13T13:35:59.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:35:59 smithi043 ceph-mon[30106]: osdmap e358: 8 total, 8 up, 8 in 2024-09-13T13:36:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:00 smithi043 ceph-mon[30106]: osdmap e359: 8 total, 8 up, 8 in 2024-09-13T13:36:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:00 smithi043 ceph-mon[40255]: osdmap e359: 8 total, 8 up, 8 in 2024-09-13T13:36:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:00 smithi189 ceph-mon[32912]: osdmap e359: 8 total, 8 up, 8 in 2024-09-13T13:36:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:01.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: pgmap v492: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: osdmap e360: 8 total, 8 up, 8 in 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-55"} : dispatch 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-55"} : dispatch 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: pgmap v492: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:36:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: osdmap e360: 8 total, 8 up, 8 in 2024-09-13T13:36:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-55"} : dispatch 2024-09-13T13:36:01.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-55"} : dispatch 2024-09-13T13:36:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: pgmap v492: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:36:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: osdmap e360: 8 total, 8 up, 8 in 2024-09-13T13:36:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:01.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:01.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-55"} : dispatch 2024-09-13T13:36:01.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-55"} : dispatch 2024-09-13T13:36:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:02 smithi043 ceph-mon[30106]: osdmap e361: 8 total, 8 up, 8 in 2024-09-13T13:36:02.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:02 smithi043 ceph-mon[40255]: osdmap e361: 8 total, 8 up, 8 in 2024-09-13T13:36:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:02 smithi189 ceph-mon[32912]: osdmap e361: 8 total, 8 up, 8 in 2024-09-13T13:36:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[30106]: pgmap v495: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:36:03.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[30106]: osdmap e362: 8 total, 8 up, 8 in 2024-09-13T13:36:03.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:03.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[40255]: pgmap v495: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:36:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[40255]: osdmap e362: 8 total, 8 up, 8 in 2024-09-13T13:36:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:03 smithi189 ceph-mon[32912]: pgmap v495: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:36:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:03 smithi189 ceph-mon[32912]: osdmap e362: 8 total, 8 up, 8 in 2024-09-13T13:36:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[30106]: osdmap e363: 8 total, 8 up, 8 in 2024-09-13T13:36:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-57"} : dispatch 2024-09-13T13:36:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-57"} : dispatch 2024-09-13T13:36:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[40255]: osdmap e363: 8 total, 8 up, 8 in 2024-09-13T13:36:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-57"} : dispatch 2024-09-13T13:36:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-57"} : dispatch 2024-09-13T13:36:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:04 smithi189 ceph-mon[32912]: osdmap e363: 8 total, 8 up, 8 in 2024-09-13T13:36:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-57"} : dispatch 2024-09-13T13:36:04.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-57"} : dispatch 2024-09-13T13:36:05.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[30106]: pgmap v498: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:05.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[30106]: osdmap e364: 8 total, 8 up, 8 in 2024-09-13T13:36:05.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:36:05.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:05.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[40255]: pgmap v498: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:05.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[40255]: osdmap e364: 8 total, 8 up, 8 in 2024-09-13T13:36:05.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:36:05.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:05.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:36:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:36:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:05 smithi189 ceph-mon[32912]: pgmap v498: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:05 smithi189 ceph-mon[32912]: osdmap e364: 8 total, 8 up, 8 in 2024-09-13T13:36:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:05 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:36:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:36:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:36:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[30106]: osdmap e365: 8 total, 8 up, 8 in 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[40255]: osdmap e365: 8 total, 8 up, 8 in 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:06 smithi189 ceph-mon[32912]: osdmap e365: 8 total, 8 up, 8 in 2024-09-13T13:36:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: pgmap v501: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: osdmap e366: 8 total, 8 up, 8 in 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-59"} : dispatch 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-59"} : dispatch 2024-09-13T13:36:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: pgmap v501: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: osdmap e366: 8 total, 8 up, 8 in 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-59"} : dispatch 2024-09-13T13:36:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-59"} : dispatch 2024-09-13T13:36:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: pgmap v501: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:36:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: osdmap e366: 8 total, 8 up, 8 in 2024-09-13T13:36:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-59"} : dispatch 2024-09-13T13:36:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-59"} : dispatch 2024-09-13T13:36:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:08 smithi043 ceph-mon[30106]: osdmap e367: 8 total, 8 up, 8 in 2024-09-13T13:36:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:08 smithi043 ceph-mon[40255]: osdmap e367: 8 total, 8 up, 8 in 2024-09-13T13:36:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:08 smithi189 ceph-mon[32912]: osdmap e367: 8 total, 8 up, 8 in 2024-09-13T13:36:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[30106]: pgmap v504: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:36:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[30106]: osdmap e368: 8 total, 8 up, 8 in 2024-09-13T13:36:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[40255]: pgmap v504: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:36:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[40255]: osdmap e368: 8 total, 8 up, 8 in 2024-09-13T13:36:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:09 smithi189 ceph-mon[32912]: pgmap v504: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:36:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:09 smithi189 ceph-mon[32912]: osdmap e368: 8 total, 8 up, 8 in 2024-09-13T13:36:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[30106]: osdmap e369: 8 total, 8 up, 8 in 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[40255]: osdmap e369: 8 total, 8 up, 8 in 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:10.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:10 smithi189 ceph-mon[32912]: osdmap e369: 8 total, 8 up, 8 in 2024-09-13T13:36:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[30106]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:36:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[30106]: osdmap e370: 8 total, 8 up, 8 in 2024-09-13T13:36:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:11.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:11.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-61"} : dispatch 2024-09-13T13:36:11.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-61"} : dispatch 2024-09-13T13:36:11.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[40255]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:36:11.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:11.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[40255]: osdmap e370: 8 total, 8 up, 8 in 2024-09-13T13:36:11.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:11.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:11.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-61"} : dispatch 2024-09-13T13:36:11.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-61"} : dispatch 2024-09-13T13:36:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:11 smithi189 ceph-mon[32912]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:36:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:11 smithi189 ceph-mon[32912]: osdmap e370: 8 total, 8 up, 8 in 2024-09-13T13:36:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:11.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:11.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-61"} : dispatch 2024-09-13T13:36:11.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-61"} : dispatch 2024-09-13T13:36:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:12 smithi043 ceph-mon[30106]: osdmap e371: 8 total, 8 up, 8 in 2024-09-13T13:36:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:12 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:12 smithi043 ceph-mon[40255]: osdmap e371: 8 total, 8 up, 8 in 2024-09-13T13:36:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:12 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:12 smithi189 ceph-mon[32912]: osdmap e371: 8 total, 8 up, 8 in 2024-09-13T13:36:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:12 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:36:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:13 smithi189 ceph-mon[32912]: pgmap v510: 196 pgs: 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:36:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:13 smithi189 ceph-mon[32912]: osdmap e372: 8 total, 8 up, 8 in 2024-09-13T13:36:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[30106]: pgmap v510: 196 pgs: 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[30106]: osdmap e372: 8 total, 8 up, 8 in 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[40255]: pgmap v510: 196 pgs: 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[40255]: osdmap e372: 8 total, 8 up, 8 in 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:14 smithi189 ceph-mon[32912]: osdmap e373: 8 total, 8 up, 8 in 2024-09-13T13:36:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[30106]: osdmap e373: 8 total, 8 up, 8 in 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[40255]: osdmap e373: 8 total, 8 up, 8 in 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:15 smithi189 ceph-mon[32912]: pgmap v513: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:15 smithi189 ceph-mon[32912]: osdmap e374: 8 total, 8 up, 8 in 2024-09-13T13:36:15.899 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:36:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:36:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:36:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:36:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:36:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:15 smithi043 ceph-mon[30106]: pgmap v513: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:15 smithi043 ceph-mon[30106]: osdmap e374: 8 total, 8 up, 8 in 2024-09-13T13:36:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:15 smithi043 ceph-mon[40255]: pgmap v513: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:15 smithi043 ceph-mon[40255]: osdmap e374: 8 total, 8 up, 8 in 2024-09-13T13:36:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:16 smithi189 ceph-mon[32912]: osdmap e375: 8 total, 8 up, 8 in 2024-09-13T13:36:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:16 smithi043 ceph-mon[30106]: osdmap e375: 8 total, 8 up, 8 in 2024-09-13T13:36:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:16 smithi043 ceph-mon[40255]: osdmap e375: 8 total, 8 up, 8 in 2024-09-13T13:36:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:17 smithi189 ceph-mon[32912]: pgmap v516: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-13T13:36:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:17 smithi189 ceph-mon[32912]: osdmap e376: 8 total, 8 up, 8 in 2024-09-13T13:36:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:17 smithi043 ceph-mon[30106]: pgmap v516: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-13T13:36:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:17 smithi043 ceph-mon[30106]: osdmap e376: 8 total, 8 up, 8 in 2024-09-13T13:36:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:17 smithi043 ceph-mon[40255]: pgmap v516: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-13T13:36:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:17 smithi043 ceph-mon[40255]: osdmap e376: 8 total, 8 up, 8 in 2024-09-13T13:36:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:18 smithi189 ceph-mon[32912]: osdmap e377: 8 total, 8 up, 8 in 2024-09-13T13:36:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:18 smithi043 ceph-mon[30106]: osdmap e377: 8 total, 8 up, 8 in 2024-09-13T13:36:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:18 smithi043 ceph-mon[40255]: osdmap e377: 8 total, 8 up, 8 in 2024-09-13T13:36:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:19 smithi189 ceph-mon[32912]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-13T13:36:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:19 smithi189 ceph-mon[32912]: osdmap e378: 8 total, 8 up, 8 in 2024-09-13T13:36:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:19 smithi043 ceph-mon[30106]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-13T13:36:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:19 smithi043 ceph-mon[30106]: osdmap e378: 8 total, 8 up, 8 in 2024-09-13T13:36:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:19 smithi043 ceph-mon[40255]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-13T13:36:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:19 smithi043 ceph-mon[40255]: osdmap e378: 8 total, 8 up, 8 in 2024-09-13T13:36:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:21.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:21 smithi189 ceph-mon[32912]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-13T13:36:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:21 smithi043 ceph-mon[30106]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-13T13:36:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:21 smithi043 ceph-mon[40255]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-13T13:36:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:22 smithi043 ceph-mon[30106]: osdmap e379: 8 total, 8 up, 8 in 2024-09-13T13:36:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:22 smithi043 ceph-mon[30106]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-13T13:36:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:22 smithi043 ceph-mon[40255]: osdmap e379: 8 total, 8 up, 8 in 2024-09-13T13:36:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:22 smithi043 ceph-mon[40255]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-13T13:36:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:22 smithi189 ceph-mon[32912]: osdmap e379: 8 total, 8 up, 8 in 2024-09-13T13:36:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:22 smithi189 ceph-mon[32912]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-13T13:36:25.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:25 smithi189 ceph-mon[32912]: pgmap v524: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 906 B/s wr, 3 op/s 2024-09-13T13:36:25.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:36:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:36:25.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:25 smithi043 ceph-mon[30106]: pgmap v524: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 906 B/s wr, 3 op/s 2024-09-13T13:36:25.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:25 smithi043 ceph-mon[40255]: pgmap v524: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 906 B/s wr, 3 op/s 2024-09-13T13:36:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:36:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:36:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:36:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:27 smithi189 ceph-mon[32912]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-13T13:36:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:27 smithi043 ceph-mon[30106]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-13T13:36:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:27 smithi043 ceph-mon[40255]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-13T13:36:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:29 smithi189 ceph-mon[32912]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 629 B/s wr, 2 op/s 2024-09-13T13:36:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:29 smithi043 ceph-mon[30106]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 629 B/s wr, 2 op/s 2024-09-13T13:36:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:29 smithi043 ceph-mon[40255]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 629 B/s wr, 2 op/s 2024-09-13T13:36:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:30 smithi189 ceph-mon[32912]: osdmap e380: 8 total, 8 up, 8 in 2024-09-13T13:36:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:30 smithi043 ceph-mon[30106]: osdmap e380: 8 total, 8 up, 8 in 2024-09-13T13:36:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:30 smithi043 ceph-mon[40255]: osdmap e380: 8 total, 8 up, 8 in 2024-09-13T13:36:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:31 smithi189 ceph-mon[32912]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:31 smithi043 ceph-mon[30106]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:31 smithi043 ceph-mon[40255]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:32 smithi043 ceph-mon[30106]: osdmap e381: 8 total, 8 up, 8 in 2024-09-13T13:36:33.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:32 smithi043 ceph-mon[30106]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:32 smithi043 ceph-mon[40255]: osdmap e381: 8 total, 8 up, 8 in 2024-09-13T13:36:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:32 smithi043 ceph-mon[40255]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:32 smithi189 ceph-mon[32912]: osdmap e381: 8 total, 8 up, 8 in 2024-09-13T13:36:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:32 smithi189 ceph-mon[32912]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:35.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:36:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:36:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:35 smithi189 ceph-mon[32912]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:35.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:35 smithi043 ceph-mon[30106]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:35.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:35 smithi043 ceph-mon[40255]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:36:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:36:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:36:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:36.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:37 smithi189 ceph-mon[32912]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:37 smithi043 ceph-mon[30106]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:37 smithi043 ceph-mon[40255]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:39.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:39 smithi189 ceph-mon[32912]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:39 smithi043 ceph-mon[30106]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:39 smithi043 ceph-mon[40255]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:36:41.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:41 smithi189 ceph-mon[32912]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:41.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:41 smithi189 ceph-mon[32912]: osdmap e382: 8 total, 8 up, 8 in 2024-09-13T13:36:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:41 smithi043 ceph-mon[30106]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:41 smithi043 ceph-mon[30106]: osdmap e382: 8 total, 8 up, 8 in 2024-09-13T13:36:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:41 smithi043 ceph-mon[40255]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:41 smithi043 ceph-mon[40255]: osdmap e382: 8 total, 8 up, 8 in 2024-09-13T13:36:43.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:43 smithi189 ceph-mon[32912]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:43 smithi043 ceph-mon[30106]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:43 smithi043 ceph-mon[40255]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:45.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:45 smithi189 ceph-mon[32912]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:45.649 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:36:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:36:45.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:45 smithi043 ceph-mon[30106]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:45.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:45 smithi043 ceph-mon[40255]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:36:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:36:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:36:46.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:46.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:47 smithi189 ceph-mon[32912]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:47.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:47 smithi189 ceph-mon[32912]: osdmap e383: 8 total, 8 up, 8 in 2024-09-13T13:36:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:47 smithi043 ceph-mon[30106]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:47 smithi043 ceph-mon[30106]: osdmap e383: 8 total, 8 up, 8 in 2024-09-13T13:36:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:47 smithi043 ceph-mon[40255]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:36:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:47 smithi043 ceph-mon[40255]: osdmap e383: 8 total, 8 up, 8 in 2024-09-13T13:36:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:49 smithi189 ceph-mon[32912]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:36:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:49 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:36:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:49 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:36:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:49 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:36:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:49 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:36:49.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[30106]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:36:49.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[40255]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:36:49.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:49 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:36:50.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:50.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:50.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:36:51.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:51 smithi189 ceph-mon[32912]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:36:51.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:51.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:51.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-63"} : dispatch 2024-09-13T13:36:51.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-63"} : dispatch 2024-09-13T13:36:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[30106]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:36:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-63"} : dispatch 2024-09-13T13:36:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-63"} : dispatch 2024-09-13T13:36:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[40255]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:36:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:36:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-63"} : dispatch 2024-09-13T13:36:51.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-63"} : dispatch 2024-09-13T13:36:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:52 smithi043 ceph-mon[30106]: osdmap e384: 8 total, 8 up, 8 in 2024-09-13T13:36:52.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:52 smithi043 ceph-mon[40255]: osdmap e384: 8 total, 8 up, 8 in 2024-09-13T13:36:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:52 smithi189 ceph-mon[32912]: osdmap e384: 8 total, 8 up, 8 in 2024-09-13T13:36:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[30106]: pgmap v543: 196 pgs: 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[30106]: osdmap e385: 8 total, 8 up, 8 in 2024-09-13T13:36:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[40255]: pgmap v543: 196 pgs: 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[40255]: osdmap e385: 8 total, 8 up, 8 in 2024-09-13T13:36:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:53 smithi189 ceph-mon[32912]: pgmap v543: 196 pgs: 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:53 smithi189 ceph-mon[32912]: osdmap e385: 8 total, 8 up, 8 in 2024-09-13T13:36:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:36:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[40255]: osdmap e386: 8 total, 8 up, 8 in 2024-09-13T13:36:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[30106]: osdmap e386: 8 total, 8 up, 8 in 2024-09-13T13:36:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:36:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:54 smithi189 ceph-mon[32912]: osdmap e386: 8 total, 8 up, 8 in 2024-09-13T13:36:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:36:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:36:55.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:55 smithi043 ceph-mon[40255]: pgmap v546: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:55.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:55.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:55 smithi043 ceph-mon[40255]: osdmap e387: 8 total, 8 up, 8 in 2024-09-13T13:36:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:55 smithi043 ceph-mon[30106]: pgmap v546: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:55.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:55 smithi043 ceph-mon[30106]: osdmap e387: 8 total, 8 up, 8 in 2024-09-13T13:36:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:55 smithi189 ceph-mon[32912]: pgmap v546: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:36:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:36:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:55 smithi189 ceph-mon[32912]: osdmap e387: 8 total, 8 up, 8 in 2024-09-13T13:36:55.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:36:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:36:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:36:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:36:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:36:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:56 smithi043 ceph-mon[30106]: osdmap e388: 8 total, 8 up, 8 in 2024-09-13T13:36:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:56 smithi043 ceph-mon[40255]: osdmap e388: 8 total, 8 up, 8 in 2024-09-13T13:36:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:56 smithi189 ceph-mon[32912]: osdmap e388: 8 total, 8 up, 8 in 2024-09-13T13:36:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:36:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:57 smithi043 ceph-mon[30106]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:36:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:57 smithi043 ceph-mon[30106]: osdmap e389: 8 total, 8 up, 8 in 2024-09-13T13:36:57.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:57 smithi043 ceph-mon[40255]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:36:57.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:57 smithi043 ceph-mon[40255]: osdmap e389: 8 total, 8 up, 8 in 2024-09-13T13:36:57.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:57 smithi189 ceph-mon[32912]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:36:57.901 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:57 smithi189 ceph-mon[32912]: osdmap e389: 8 total, 8 up, 8 in 2024-09-13T13:36:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:58 smithi043 ceph-mon[30106]: osdmap e390: 8 total, 8 up, 8 in 2024-09-13T13:36:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:58 smithi043 ceph-mon[40255]: osdmap e390: 8 total, 8 up, 8 in 2024-09-13T13:36:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:58 smithi189 ceph-mon[32912]: osdmap e390: 8 total, 8 up, 8 in 2024-09-13T13:36:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:36:59 smithi043 ceph-mon[30106]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:36:59.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:36:59 smithi043 ceph-mon[40255]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:36:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:36:59 smithi189 ceph-mon[32912]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:37:01.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:01 smithi043 ceph-mon[30106]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-13T13:37:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:01 smithi043 ceph-mon[40255]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-13T13:37:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:01 smithi189 ceph-mon[32912]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-13T13:37:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:02 smithi043 ceph-mon[40255]: osdmap e391: 8 total, 8 up, 8 in 2024-09-13T13:37:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:02 smithi043 ceph-mon[40255]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-13T13:37:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:02 smithi043 ceph-mon[30106]: osdmap e391: 8 total, 8 up, 8 in 2024-09-13T13:37:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:02 smithi043 ceph-mon[30106]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-13T13:37:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:02 smithi189 ceph-mon[32912]: osdmap e391: 8 total, 8 up, 8 in 2024-09-13T13:37:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:02 smithi189 ceph-mon[32912]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-13T13:37:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:05 smithi189 ceph-mon[32912]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 519 B/s wr, 3 op/s 2024-09-13T13:37:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:05.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:37:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:37:05.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:05 smithi043 ceph-mon[30106]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 519 B/s wr, 3 op/s 2024-09-13T13:37:05.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:05.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:05 smithi043 ceph-mon[40255]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 519 B/s wr, 3 op/s 2024-09-13T13:37:05.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:37:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:37:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:37:06.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:07.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:07 smithi189 ceph-mon[32912]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 461 B/s wr, 3 op/s 2024-09-13T13:37:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:07 smithi043 ceph-mon[30106]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 461 B/s wr, 3 op/s 2024-09-13T13:37:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:07 smithi043 ceph-mon[40255]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 461 B/s wr, 3 op/s 2024-09-13T13:37:08.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:08.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:08.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-65"} : dispatch 2024-09-13T13:37:08.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-65"} : dispatch 2024-09-13T13:37:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:08.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-65"} : dispatch 2024-09-13T13:37:08.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-65"} : dispatch 2024-09-13T13:37:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-65"} : dispatch 2024-09-13T13:37:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-65"} : dispatch 2024-09-13T13:37:09.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:09 smithi189 ceph-mon[32912]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-13T13:37:09.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:09 smithi189 ceph-mon[32912]: osdmap e392: 8 total, 8 up, 8 in 2024-09-13T13:37:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:09 smithi043 ceph-mon[30106]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-13T13:37:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:09 smithi043 ceph-mon[30106]: osdmap e392: 8 total, 8 up, 8 in 2024-09-13T13:37:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:09 smithi043 ceph-mon[40255]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-13T13:37:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:09 smithi043 ceph-mon[40255]: osdmap e392: 8 total, 8 up, 8 in 2024-09-13T13:37:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:10 smithi189 ceph-mon[32912]: osdmap e393: 8 total, 8 up, 8 in 2024-09-13T13:37:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:10 smithi043 ceph-mon[30106]: osdmap e393: 8 total, 8 up, 8 in 2024-09-13T13:37:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:10 smithi043 ceph-mon[40255]: osdmap e393: 8 total, 8 up, 8 in 2024-09-13T13:37:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:11.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:11 smithi189 ceph-mon[32912]: pgmap v561: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:11.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:11 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:11.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:11.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:11 smithi189 ceph-mon[32912]: osdmap e394: 8 total, 8 up, 8 in 2024-09-13T13:37:11.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[30106]: pgmap v561: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[30106]: osdmap e394: 8 total, 8 up, 8 in 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[40255]: pgmap v561: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[40255]: osdmap e394: 8 total, 8 up, 8 in 2024-09-13T13:37:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[30106]: osdmap e395: 8 total, 8 up, 8 in 2024-09-13T13:37:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-67"} : dispatch 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-67"} : dispatch 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[40255]: osdmap e395: 8 total, 8 up, 8 in 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-67"} : dispatch 2024-09-13T13:37:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-67"} : dispatch 2024-09-13T13:37:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:12 smithi189 ceph-mon[32912]: osdmap e395: 8 total, 8 up, 8 in 2024-09-13T13:37:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-67"} : dispatch 2024-09-13T13:37:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-67"} : dispatch 2024-09-13T13:37:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:13 smithi043 ceph-mon[30106]: pgmap v564: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:13 smithi043 ceph-mon[30106]: osdmap e396: 8 total, 8 up, 8 in 2024-09-13T13:37:13.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:13 smithi043 ceph-mon[40255]: pgmap v564: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:13.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:13 smithi043 ceph-mon[40255]: osdmap e396: 8 total, 8 up, 8 in 2024-09-13T13:37:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:13 smithi189 ceph-mon[32912]: pgmap v564: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:13 smithi189 ceph-mon[32912]: osdmap e396: 8 total, 8 up, 8 in 2024-09-13T13:37:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:14 smithi043 ceph-mon[30106]: osdmap e397: 8 total, 8 up, 8 in 2024-09-13T13:37:14.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:14.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:14.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:14 smithi043 ceph-mon[40255]: osdmap e397: 8 total, 8 up, 8 in 2024-09-13T13:37:14.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:14.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:14 smithi189 ceph-mon[32912]: osdmap e397: 8 total, 8 up, 8 in 2024-09-13T13:37:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:14.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:15.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: pgmap v567: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:15.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: osdmap e398: 8 total, 8 up, 8 in 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-69"} : dispatch 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-69"} : dispatch 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: pgmap v567: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: osdmap e398: 8 total, 8 up, 8 in 2024-09-13T13:37:15.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:15.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:15.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:15.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-69"} : dispatch 2024-09-13T13:37:15.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-69"} : dispatch 2024-09-13T13:37:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: pgmap v567: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: osdmap e398: 8 total, 8 up, 8 in 2024-09-13T13:37:15.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:15.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:15.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:15.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-69"} : dispatch 2024-09-13T13:37:15.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-69"} : dispatch 2024-09-13T13:37:15.899 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:37:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:37:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:37:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:37:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:37:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:16 smithi043 ceph-mon[30106]: osdmap e399: 8 total, 8 up, 8 in 2024-09-13T13:37:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:16.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:16 smithi043 ceph-mon[40255]: osdmap e399: 8 total, 8 up, 8 in 2024-09-13T13:37:16.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:16 smithi189 ceph-mon[32912]: osdmap e399: 8 total, 8 up, 8 in 2024-09-13T13:37:16.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[30106]: pgmap v570: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:37:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[30106]: osdmap e400: 8 total, 8 up, 8 in 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[40255]: pgmap v570: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[40255]: osdmap e400: 8 total, 8 up, 8 in 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:17 smithi189 ceph-mon[32912]: pgmap v570: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:37:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:17 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:17 smithi189 ceph-mon[32912]: osdmap e400: 8 total, 8 up, 8 in 2024-09-13T13:37:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[30106]: osdmap e401: 8 total, 8 up, 8 in 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[40255]: osdmap e401: 8 total, 8 up, 8 in 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:37:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:37:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:18 smithi189 ceph-mon[32912]: osdmap e401: 8 total, 8 up, 8 in 2024-09-13T13:37:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:37:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:37:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:19 smithi043 ceph-mon[30106]: pgmap v573: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:37:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:37:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:19 smithi043 ceph-mon[30106]: osdmap e402: 8 total, 8 up, 8 in 2024-09-13T13:37:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:19 smithi043 ceph-mon[40255]: pgmap v573: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:37:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:37:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:19 smithi043 ceph-mon[40255]: osdmap e402: 8 total, 8 up, 8 in 2024-09-13T13:37:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:19 smithi189 ceph-mon[32912]: pgmap v573: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:37:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:37:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:19 smithi189 ceph-mon[32912]: osdmap e402: 8 total, 8 up, 8 in 2024-09-13T13:37:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:20 smithi043 ceph-mon[40255]: osdmap e403: 8 total, 8 up, 8 in 2024-09-13T13:37:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:20 smithi043 ceph-mon[30106]: osdmap e403: 8 total, 8 up, 8 in 2024-09-13T13:37:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:20 smithi189 ceph-mon[32912]: osdmap e403: 8 total, 8 up, 8 in 2024-09-13T13:37:20.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:21 smithi043 ceph-mon[30106]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-13T13:37:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:21 smithi043 ceph-mon[40255]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-13T13:37:21.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:21 smithi189 ceph-mon[32912]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-13T13:37:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:23 smithi043 ceph-mon[30106]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-13T13:37:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:23 smithi043 ceph-mon[40255]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-13T13:37:23.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:23 smithi189 ceph-mon[32912]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-13T13:37:25.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:25 smithi043 ceph-mon[30106]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 893 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-13T13:37:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:25 smithi043 ceph-mon[40255]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 893 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-13T13:37:25.744 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:37:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:37:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:37:25.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:25 smithi189 ceph-mon[32912]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 893 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-13T13:37:25.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:37:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:37:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:26.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:27 smithi043 ceph-mon[30106]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-13T13:37:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:27 smithi043 ceph-mon[40255]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-13T13:37:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:27 smithi189 ceph-mon[32912]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-13T13:37:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:29 smithi043 ceph-mon[30106]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-13T13:37:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:29 smithi043 ceph-mon[40255]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-13T13:37:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:29 smithi189 ceph-mon[32912]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-13T13:37:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:31 smithi043 ceph-mon[30106]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-13T13:37:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:31 smithi043 ceph-mon[40255]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-13T13:37:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:31 smithi189 ceph-mon[32912]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-13T13:37:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:33 smithi043 ceph-mon[30106]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:37:33.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:33 smithi043 ceph-mon[40255]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:37:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:33 smithi189 ceph-mon[32912]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:37:35.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:37:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:37:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:35 smithi189 ceph-mon[32912]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:37:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]} : dispatch 2024-09-13T13:37:35.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]} : dispatch 2024-09-13T13:37:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:37:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[30106]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]} : dispatch 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]} : dispatch 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[40255]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]} : dispatch 2024-09-13T13:37:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]} : dispatch 2024-09-13T13:37:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:36 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]}]': finished 2024-09-13T13:37:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:36 smithi189 ceph-mon[32912]: osdmap e404: 8 total, 8 up, 8 in 2024-09-13T13:37:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:36 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]}]': finished 2024-09-13T13:37:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:36 smithi043 ceph-mon[30106]: osdmap e404: 8 total, 8 up, 8 in 2024-09-13T13:37:36.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:36 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.0", "id": [6, 3]}]': finished 2024-09-13T13:37:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:36 smithi043 ceph-mon[40255]: osdmap e404: 8 total, 8 up, 8 in 2024-09-13T13:37:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:37 smithi189 ceph-mon[32912]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-13T13:37:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:37 smithi189 ceph-mon[32912]: osdmap e405: 8 total, 8 up, 8 in 2024-09-13T13:37:37.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:37 smithi043 ceph-mon[30106]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-13T13:37:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:37 smithi043 ceph-mon[30106]: osdmap e405: 8 total, 8 up, 8 in 2024-09-13T13:37:37.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:37 smithi043 ceph-mon[40255]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-13T13:37:37.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:37 smithi043 ceph-mon[40255]: osdmap e405: 8 total, 8 up, 8 in 2024-09-13T13:37:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:39 smithi189 ceph-mon[32912]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:37:39.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:39 smithi043 ceph-mon[30106]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:37:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:39 smithi043 ceph-mon[40255]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:37:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-71"} : dispatch 2024-09-13T13:37:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-71"} : dispatch 2024-09-13T13:37:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-71"} : dispatch 2024-09-13T13:37:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-71"} : dispatch 2024-09-13T13:37:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-71"} : dispatch 2024-09-13T13:37:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-71"} : dispatch 2024-09-13T13:37:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:41 smithi189 ceph-mon[32912]: pgmap v588: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:37:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:41 smithi189 ceph-mon[32912]: osdmap e406: 8 total, 8 up, 8 in 2024-09-13T13:37:41.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:41 smithi043 ceph-mon[30106]: pgmap v588: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:37:41.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:41 smithi043 ceph-mon[30106]: osdmap e406: 8 total, 8 up, 8 in 2024-09-13T13:37:41.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:41 smithi043 ceph-mon[40255]: pgmap v588: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:37:41.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:41 smithi043 ceph-mon[40255]: osdmap e406: 8 total, 8 up, 8 in 2024-09-13T13:37:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:42 smithi189 ceph-mon[32912]: osdmap e407: 8 total, 8 up, 8 in 2024-09-13T13:37:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:42.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:42 smithi043 ceph-mon[30106]: osdmap e407: 8 total, 8 up, 8 in 2024-09-13T13:37:42.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:42.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:42 smithi043 ceph-mon[40255]: osdmap e407: 8 total, 8 up, 8 in 2024-09-13T13:37:42.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:42.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:43 smithi189 ceph-mon[32912]: pgmap v591: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:43 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:43 smithi189 ceph-mon[32912]: osdmap e408: 8 total, 8 up, 8 in 2024-09-13T13:37:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[30106]: pgmap v591: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[30106]: osdmap e408: 8 total, 8 up, 8 in 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[40255]: pgmap v591: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[40255]: osdmap e408: 8 total, 8 up, 8 in 2024-09-13T13:37:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:44 smithi189 ceph-mon[32912]: osdmap e409: 8 total, 8 up, 8 in 2024-09-13T13:37:44.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:44 smithi043 ceph-mon[30106]: osdmap e409: 8 total, 8 up, 8 in 2024-09-13T13:37:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:44 smithi043 ceph-mon[40255]: osdmap e409: 8 total, 8 up, 8 in 2024-09-13T13:37:45.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:45 smithi189 ceph-mon[32912]: pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:45.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:45 smithi189 ceph-mon[32912]: osdmap e410: 8 total, 8 up, 8 in 2024-09-13T13:37:45.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:37:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:37:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:37:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:37:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:37:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:45 smithi043 ceph-mon[40255]: pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:45 smithi043 ceph-mon[40255]: osdmap e410: 8 total, 8 up, 8 in 2024-09-13T13:37:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:45 smithi043 ceph-mon[30106]: pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:37:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:45 smithi043 ceph-mon[30106]: osdmap e410: 8 total, 8 up, 8 in 2024-09-13T13:37:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:46.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:46.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:47 smithi189 ceph-mon[32912]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-13T13:37:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:47 smithi043 ceph-mon[30106]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-13T13:37:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:47 smithi043 ceph-mon[40255]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-13T13:37:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:49 smithi189 ceph-mon[32912]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:37:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:49 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:37:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:49 smithi043 ceph-mon[30106]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:37:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:49 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:37:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:49 smithi043 ceph-mon[40255]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-13T13:37:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:49 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:37:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:37:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:37:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:50 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:37:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:37:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:37:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:37:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:37:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:37:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:37:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:37:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:51 smithi043 ceph-mon[30106]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 919 B/s wr, 4 op/s 2024-09-13T13:37:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:51 smithi043 ceph-mon[40255]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 919 B/s wr, 4 op/s 2024-09-13T13:37:52.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:51 smithi189 ceph-mon[32912]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 919 B/s wr, 4 op/s 2024-09-13T13:37:52.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:52 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:52 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:52 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:53 smithi043 ceph-mon[30106]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 815 B/s wr, 3 op/s 2024-09-13T13:37:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:53 smithi043 ceph-mon[40255]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 815 B/s wr, 3 op/s 2024-09-13T13:37:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:53 smithi189 ceph-mon[32912]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 815 B/s wr, 3 op/s 2024-09-13T13:37:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-73"} : dispatch 2024-09-13T13:37:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-73"} : dispatch 2024-09-13T13:37:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-73"} : dispatch 2024-09-13T13:37:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-73"} : dispatch 2024-09-13T13:37:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:37:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-73"} : dispatch 2024-09-13T13:37:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-73"} : dispatch 2024-09-13T13:37:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:55 smithi189 ceph-mon[32912]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-13T13:37:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:55 smithi189 ceph-mon[32912]: osdmap e411: 8 total, 8 up, 8 in 2024-09-13T13:37:55.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:37:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:37:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:37:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:37:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:37:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:55 smithi043 ceph-mon[30106]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-13T13:37:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:55 smithi043 ceph-mon[30106]: osdmap e411: 8 total, 8 up, 8 in 2024-09-13T13:37:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:55 smithi043 ceph-mon[40255]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-13T13:37:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:55 smithi043 ceph-mon[40255]: osdmap e411: 8 total, 8 up, 8 in 2024-09-13T13:37:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[30106]: osdmap e412: 8 total, 8 up, 8 in 2024-09-13T13:37:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[40255]: osdmap e412: 8 total, 8 up, 8 in 2024-09-13T13:37:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:37:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:56 smithi189 ceph-mon[32912]: osdmap e412: 8 total, 8 up, 8 in 2024-09-13T13:37:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:37:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:57 smithi189 ceph-mon[32912]: pgmap v603: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:37:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:58.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:57 smithi189 ceph-mon[32912]: osdmap e413: 8 total, 8 up, 8 in 2024-09-13T13:37:58.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:58.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:57 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:58.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[30106]: pgmap v603: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[30106]: osdmap e413: 8 total, 8 up, 8 in 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[40255]: pgmap v603: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[40255]: osdmap e413: 8 total, 8 up, 8 in 2024-09-13T13:37:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:37:58.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:57 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:37:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:58 smithi189 ceph-mon[32912]: osdmap e414: 8 total, 8 up, 8 in 2024-09-13T13:37:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:58 smithi043 ceph-mon[30106]: osdmap e414: 8 total, 8 up, 8 in 2024-09-13T13:37:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:58 smithi043 ceph-mon[40255]: osdmap e414: 8 total, 8 up, 8 in 2024-09-13T13:38:00.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:59 smithi189 ceph-mon[32912]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:59 smithi189 ceph-mon[32912]: osdmap e415: 8 total, 8 up, 8 in 2024-09-13T13:38:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-75"} : dispatch 2024-09-13T13:38:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:37:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-75"} : dispatch 2024-09-13T13:38:00.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[30106]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[30106]: osdmap e415: 8 total, 8 up, 8 in 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-75"} : dispatch 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-75"} : dispatch 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[40255]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[40255]: osdmap e415: 8 total, 8 up, 8 in 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-75"} : dispatch 2024-09-13T13:38:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:37:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-75"} : dispatch 2024-09-13T13:38:01.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:00 smithi189 ceph-mon[32912]: osdmap e416: 8 total, 8 up, 8 in 2024-09-13T13:38:01.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:00 smithi043 ceph-mon[30106]: osdmap e416: 8 total, 8 up, 8 in 2024-09-13T13:38:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:00 smithi043 ceph-mon[40255]: osdmap e416: 8 total, 8 up, 8 in 2024-09-13T13:38:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:01 smithi189 ceph-mon[32912]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:38:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:01 smithi189 ceph-mon[32912]: osdmap e417: 8 total, 8 up, 8 in 2024-09-13T13:38:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:02.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[30106]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:38:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[30106]: osdmap e417: 8 total, 8 up, 8 in 2024-09-13T13:38:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[40255]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:38:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[40255]: osdmap e417: 8 total, 8 up, 8 in 2024-09-13T13:38:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:03.143 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:03.143 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[40255]: osdmap e418: 8 total, 8 up, 8 in 2024-09-13T13:38:03.143 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:03.143 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:03.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[30106]: osdmap e418: 8 total, 8 up, 8 in 2024-09-13T13:38:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:03.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:02 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:02 smithi189 ceph-mon[32912]: osdmap e418: 8 total, 8 up, 8 in 2024-09-13T13:38:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:02 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:03 smithi189 ceph-mon[32912]: pgmap v612: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:38:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:03 smithi189 ceph-mon[32912]: osdmap e419: 8 total, 8 up, 8 in 2024-09-13T13:38:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-77"} : dispatch 2024-09-13T13:38:04.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-77"} : dispatch 2024-09-13T13:38:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[40255]: pgmap v612: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:38:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[40255]: osdmap e419: 8 total, 8 up, 8 in 2024-09-13T13:38:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:04.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-77"} : dispatch 2024-09-13T13:38:04.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-77"} : dispatch 2024-09-13T13:38:04.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[30106]: pgmap v612: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:38:04.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[30106]: osdmap e419: 8 total, 8 up, 8 in 2024-09-13T13:38:04.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:04.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:04.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-77"} : dispatch 2024-09-13T13:38:04.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-77"} : dispatch 2024-09-13T13:38:05.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:04 smithi189 ceph-mon[32912]: osdmap e420: 8 total, 8 up, 8 in 2024-09-13T13:38:05.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:04 smithi043 ceph-mon[30106]: osdmap e420: 8 total, 8 up, 8 in 2024-09-13T13:38:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:04 smithi043 ceph-mon[40255]: osdmap e420: 8 total, 8 up, 8 in 2024-09-13T13:38:05.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:38:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:38:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:05 smithi189 ceph-mon[32912]: pgmap v615: 196 pgs: 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:05 smithi189 ceph-mon[32912]: osdmap e421: 8 total, 8 up, 8 in 2024-09-13T13:38:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:38:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[30106]: pgmap v615: 196 pgs: 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[30106]: osdmap e421: 8 total, 8 up, 8 in 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[40255]: pgmap v615: 196 pgs: 196 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[40255]: osdmap e421: 8 total, 8 up, 8 in 2024-09-13T13:38:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:06 smithi189 ceph-mon[32912]: osdmap e422: 8 total, 8 up, 8 in 2024-09-13T13:38:07.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[30106]: osdmap e422: 8 total, 8 up, 8 in 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[40255]: osdmap e422: 8 total, 8 up, 8 in 2024-09-13T13:38:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:07 smithi189 ceph-mon[32912]: pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:38:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:07 smithi189 ceph-mon[32912]: osdmap e423: 8 total, 8 up, 8 in 2024-09-13T13:38:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:07 smithi043 ceph-mon[30106]: pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:38:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:07 smithi043 ceph-mon[30106]: osdmap e423: 8 total, 8 up, 8 in 2024-09-13T13:38:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:07 smithi043 ceph-mon[40255]: pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:38:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:07 smithi043 ceph-mon[40255]: osdmap e423: 8 total, 8 up, 8 in 2024-09-13T13:38:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:08 smithi189 ceph-mon[32912]: osdmap e424: 8 total, 8 up, 8 in 2024-09-13T13:38:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.0"} : dispatch 2024-09-13T13:38:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:08 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.0"}]: dispatch 2024-09-13T13:38:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:08 smithi189 ceph-mon[32912]: 304.0 deep-scrub starts 2024-09-13T13:38:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:08 smithi189 ceph-mon[32912]: 304.0 deep-scrub ok 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[30106]: osdmap e424: 8 total, 8 up, 8 in 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.0"} : dispatch 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.0"}]: dispatch 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[30106]: 304.0 deep-scrub starts 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[30106]: 304.0 deep-scrub ok 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[40255]: osdmap e424: 8 total, 8 up, 8 in 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.0"} : dispatch 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.0"}]: dispatch 2024-09-13T13:38:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[40255]: 304.0 deep-scrub starts 2024-09-13T13:38:09.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:08 smithi043 ceph-mon[40255]: 304.0 deep-scrub ok 2024-09-13T13:38:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:09 smithi189 ceph-mon[32912]: pgmap v621: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:38:10.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:09 smithi043 ceph-mon[30106]: pgmap v621: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:38:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:09 smithi043 ceph-mon[40255]: pgmap v621: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:38:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:11 smithi189 ceph-mon[32912]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-13T13:38:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:11 smithi043 ceph-mon[30106]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-13T13:38:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:11 smithi043 ceph-mon[40255]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-13T13:38:14.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:13 smithi189 ceph-mon[32912]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 784 B/s wr, 2 op/s 2024-09-13T13:38:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:13 smithi043 ceph-mon[30106]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 784 B/s wr, 2 op/s 2024-09-13T13:38:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:13 smithi043 ceph-mon[40255]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 784 B/s wr, 2 op/s 2024-09-13T13:38:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:15 smithi189 ceph-mon[32912]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-13T13:38:15.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:38:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:38:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:15 smithi043 ceph-mon[30106]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-13T13:38:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:15 smithi043 ceph-mon[40255]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-13T13:38:15.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:38:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:38:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:38:17.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:18.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:17 smithi189 ceph-mon[32912]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 537 B/s wr, 2 op/s 2024-09-13T13:38:18.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:17 smithi043 ceph-mon[30106]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 537 B/s wr, 2 op/s 2024-09-13T13:38:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:17 smithi043 ceph-mon[40255]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 537 B/s wr, 2 op/s 2024-09-13T13:38:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:19 smithi189 ceph-mon[32912]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 486 B/s wr, 1 op/s 2024-09-13T13:38:20.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:19 smithi043 ceph-mon[30106]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 486 B/s wr, 1 op/s 2024-09-13T13:38:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:19 smithi043 ceph-mon[40255]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 486 B/s wr, 1 op/s 2024-09-13T13:38:21.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:21.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:20 smithi189 ceph-mon[32912]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-13T13:38:21.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:20 smithi043 ceph-mon[30106]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-13T13:38:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:20 smithi043 ceph-mon[40255]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-13T13:38:23.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:23 smithi189 ceph-mon[32912]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:38:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:23 smithi043 ceph-mon[30106]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:38:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:23 smithi043 ceph-mon[40255]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:38:25.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:25 smithi189 ceph-mon[32912]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:38:25.648 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:38:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:38:25.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:25 smithi043 ceph-mon[30106]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:38:25.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:25 smithi043 ceph-mon[40255]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:38:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:38:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:38:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:38:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:27 smithi189 ceph-mon[32912]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:38:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:27 smithi043 ceph-mon[40255]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:38:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:27 smithi043 ceph-mon[30106]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:38:28.148 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:38:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:38:27.845+0000 7fcae2135640 -1 snap_mapper.add_oid found existing snaps mapped on 304:06f00a8a:test-rados-api-smithi043-76627-80::foo:2, removing 2024-09-13T13:38:28.244 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:38:27 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:38:27.845+0000 7f1d96e92640 -1 snap_mapper.add_oid found existing snaps mapped on 304:06f00a8a:test-rados-api-smithi043-76627-80::foo:2, removing 2024-09-13T13:38:28.244 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:38:27 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2[56012]: 2024-09-13T13:38:27.845+0000 7f6a1a70b640 -1 snap_mapper.add_oid found existing snaps mapped on 304:06f00a8a:test-rados-api-smithi043-76627-80::foo:2, removing 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-79"} : dispatch 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-79"} : dispatch 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-79"} : dispatch 2024-09-13T13:38:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-79"} : dispatch 2024-09-13T13:38:28.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:28.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:28.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-79"} : dispatch 2024-09-13T13:38:28.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-79"} : dispatch 2024-09-13T13:38:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:29 smithi043 ceph-mon[30106]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:38:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:29 smithi043 ceph-mon[30106]: osdmap e425: 8 total, 8 up, 8 in 2024-09-13T13:38:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:29 smithi043 ceph-mon[40255]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:38:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:29 smithi043 ceph-mon[40255]: osdmap e425: 8 total, 8 up, 8 in 2024-09-13T13:38:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:29 smithi189 ceph-mon[32912]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:38:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:29 smithi189 ceph-mon[32912]: osdmap e425: 8 total, 8 up, 8 in 2024-09-13T13:38:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:30 smithi043 ceph-mon[30106]: osdmap e426: 8 total, 8 up, 8 in 2024-09-13T13:38:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:30 smithi043 ceph-mon[40255]: osdmap e426: 8 total, 8 up, 8 in 2024-09-13T13:38:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:30.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:30 smithi189 ceph-mon[32912]: osdmap e426: 8 total, 8 up, 8 in 2024-09-13T13:38:30.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:30.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[30106]: pgmap v634: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[30106]: osdmap e427: 8 total, 8 up, 8 in 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[40255]: pgmap v634: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:31.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[40255]: osdmap e427: 8 total, 8 up, 8 in 2024-09-13T13:38:31.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:31.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:31.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:31 smithi189 ceph-mon[32912]: pgmap v634: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-13T13:38:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:31 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:31 smithi189 ceph-mon[32912]: osdmap e427: 8 total, 8 up, 8 in 2024-09-13T13:38:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:31.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[30106]: osdmap e428: 8 total, 8 up, 8 in 2024-09-13T13:38:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[40255]: osdmap e428: 8 total, 8 up, 8 in 2024-09-13T13:38:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:32 smithi189 ceph-mon[32912]: osdmap e428: 8 total, 8 up, 8 in 2024-09-13T13:38:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:32.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[30106]: pgmap v637: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[30106]: osdmap e429: 8 total, 8 up, 8 in 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[40255]: pgmap v637: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[40255]: osdmap e429: 8 total, 8 up, 8 in 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:33.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:33 smithi189 ceph-mon[32912]: pgmap v637: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-13T13:38:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:33 smithi189 ceph-mon[32912]: osdmap e429: 8 total, 8 up, 8 in 2024-09-13T13:38:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:33.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[30106]: osdmap e430: 8 total, 8 up, 8 in 2024-09-13T13:38:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:34.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:34.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[40255]: osdmap e430: 8 total, 8 up, 8 in 2024-09-13T13:38:34.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:34.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:34 smithi189 ceph-mon[32912]: osdmap e430: 8 total, 8 up, 8 in 2024-09-13T13:38:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:34.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[30106]: pgmap v640: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[30106]: osdmap e431: 8 total, 8 up, 8 in 2024-09-13T13:38:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[40255]: pgmap v640: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[40255]: osdmap e431: 8 total, 8 up, 8 in 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-13T13:38:35.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:35.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:38:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:38:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:35 smithi189 ceph-mon[32912]: pgmap v640: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:35.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:35.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:35 smithi189 ceph-mon[32912]: osdmap e431: 8 total, 8 up, 8 in 2024-09-13T13:38:35.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-13T13:38:35.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-13T13:38:35.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:38:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:38:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[40255]: osdmap e432: 8 total, 8 up, 8 in 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[30106]: osdmap e432: 8 total, 8 up, 8 in 2024-09-13T13:38:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-13T13:38:36.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-13T13:38:36.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-13T13:38:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:36 smithi189 ceph-mon[32912]: osdmap e432: 8 total, 8 up, 8 in 2024-09-13T13:38:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-13T13:38:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-13T13:38:36.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[30106]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[30106]: osdmap e433: 8 total, 8 up, 8 in 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[40255]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[40255]: osdmap e433: 8 total, 8 up, 8 in 2024-09-13T13:38:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:37.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:37.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:37 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:37 smithi189 ceph-mon[32912]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-13T13:38:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-13T13:38:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:37 smithi189 ceph-mon[32912]: osdmap e433: 8 total, 8 up, 8 in 2024-09-13T13:38:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:37 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[40255]: osdmap e434: 8 total, 8 up, 8 in 2024-09-13T13:38:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-81"} : dispatch 2024-09-13T13:38:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-81"} : dispatch 2024-09-13T13:38:38.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:38.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[30106]: osdmap e434: 8 total, 8 up, 8 in 2024-09-13T13:38:38.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:38.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:38.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-81"} : dispatch 2024-09-13T13:38:38.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-81"} : dispatch 2024-09-13T13:38:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:38 smithi189 ceph-mon[32912]: osdmap e434: 8 total, 8 up, 8 in 2024-09-13T13:38:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-81"} : dispatch 2024-09-13T13:38:38.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-81"} : dispatch 2024-09-13T13:38:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:39 smithi043 ceph-mon[30106]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-13T13:38:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:39 smithi043 ceph-mon[30106]: osdmap e435: 8 total, 8 up, 8 in 2024-09-13T13:38:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:39 smithi043 ceph-mon[40255]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-13T13:38:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:39 smithi043 ceph-mon[40255]: osdmap e435: 8 total, 8 up, 8 in 2024-09-13T13:38:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:39 smithi189 ceph-mon[32912]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-13T13:38:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:39 smithi189 ceph-mon[32912]: osdmap e435: 8 total, 8 up, 8 in 2024-09-13T13:38:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:40 smithi043 ceph-mon[30106]: osdmap e436: 8 total, 8 up, 8 in 2024-09-13T13:38:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:40 smithi043 ceph-mon[40255]: osdmap e436: 8 total, 8 up, 8 in 2024-09-13T13:38:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:40 smithi189 ceph-mon[32912]: osdmap e436: 8 total, 8 up, 8 in 2024-09-13T13:38:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[30106]: pgmap v649: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[30106]: osdmap e437: 8 total, 8 up, 8 in 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[40255]: pgmap v649: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[40255]: osdmap e437: 8 total, 8 up, 8 in 2024-09-13T13:38:41.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:41.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:41.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:41 smithi189 ceph-mon[32912]: pgmap v649: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-09-13T13:38:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:41 smithi189 ceph-mon[32912]: osdmap e437: 8 total, 8 up, 8 in 2024-09-13T13:38:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:41.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:42 smithi189 ceph-mon[32912]: osdmap e438: 8 total, 8 up, 8 in 2024-09-13T13:38:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:42 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:42.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:42.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:42 smithi189 ceph-mon[32912]: osdmap e439: 8 total, 8 up, 8 in 2024-09-13T13:38:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[30106]: osdmap e438: 8 total, 8 up, 8 in 2024-09-13T13:38:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[30106]: osdmap e439: 8 total, 8 up, 8 in 2024-09-13T13:38:42.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:42.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[40255]: osdmap e438: 8 total, 8 up, 8 in 2024-09-13T13:38:42.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:42.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:42.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:42.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:42.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:42 smithi043 ceph-mon[40255]: osdmap e439: 8 total, 8 up, 8 in 2024-09-13T13:38:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:43 smithi189 ceph-mon[32912]: pgmap v652: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-09-13T13:38:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:43 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:43 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:43 smithi189 ceph-mon[32912]: osdmap e440: 8 total, 8 up, 8 in 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[30106]: pgmap v652: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[30106]: osdmap e440: 8 total, 8 up, 8 in 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[40255]: pgmap v652: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:43.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:43 smithi043 ceph-mon[40255]: osdmap e440: 8 total, 8 up, 8 in 2024-09-13T13:38:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:44.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:44.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:44 smithi189 ceph-mon[32912]: osdmap e441: 8 total, 8 up, 8 in 2024-09-13T13:38:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[40255]: osdmap e441: 8 total, 8 up, 8 in 2024-09-13T13:38:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:44 smithi043 ceph-mon[30106]: osdmap e441: 8 total, 8 up, 8 in 2024-09-13T13:38:45.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:45 smithi189 ceph-mon[32912]: pgmap v655: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:45.901 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:38:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:38:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:45 smithi043 ceph-mon[30106]: pgmap v655: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:45.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:38:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:38:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:38:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:45 smithi043 ceph-mon[40255]: pgmap v655: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:46 smithi189 ceph-mon[32912]: osdmap e442: 8 total, 8 up, 8 in 2024-09-13T13:38:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:46.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:46 smithi043 ceph-mon[40255]: osdmap e442: 8 total, 8 up, 8 in 2024-09-13T13:38:46.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:46.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:46 smithi043 ceph-mon[30106]: osdmap e442: 8 total, 8 up, 8 in 2024-09-13T13:38:46.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:47 smithi189 ceph-mon[32912]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-13T13:38:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:47 smithi189 ceph-mon[32912]: osdmap e443: 8 total, 8 up, 8 in 2024-09-13T13:38:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-83"} : dispatch 2024-09-13T13:38:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-83"} : dispatch 2024-09-13T13:38:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:47 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[40255]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[40255]: osdmap e443: 8 total, 8 up, 8 in 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-83"} : dispatch 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-83"} : dispatch 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[30106]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-13T13:38:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[30106]: osdmap e443: 8 total, 8 up, 8 in 2024-09-13T13:38:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-83"} : dispatch 2024-09-13T13:38:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-83"} : dispatch 2024-09-13T13:38:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:47 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:48 smithi189 ceph-mon[32912]: osdmap e444: 8 total, 8 up, 8 in 2024-09-13T13:38:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:48 smithi189 ceph-mon[32912]: osdmap e445: 8 total, 8 up, 8 in 2024-09-13T13:38:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:48.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:48.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[40255]: osdmap e444: 8 total, 8 up, 8 in 2024-09-13T13:38:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[40255]: osdmap e445: 8 total, 8 up, 8 in 2024-09-13T13:38:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[30106]: osdmap e444: 8 total, 8 up, 8 in 2024-09-13T13:38:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[30106]: osdmap e445: 8 total, 8 up, 8 in 2024-09-13T13:38:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:49 smithi189 ceph-mon[32912]: pgmap v661: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:38:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:49 smithi189 ceph-mon[32912]: osdmap e446: 8 total, 8 up, 8 in 2024-09-13T13:38:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[30106]: pgmap v661: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[30106]: osdmap e446: 8 total, 8 up, 8 in 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[40255]: pgmap v661: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[40255]: osdmap e446: 8 total, 8 up, 8 in 2024-09-13T13:38:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:38:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:38:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:38:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:38:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:38:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:50.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: osdmap e447: 8 total, 8 up, 8 in 2024-09-13T13:38:50.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:50.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: osdmap e447: 8 total, 8 up, 8 in 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: osdmap e447: 8 total, 8 up, 8 in 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:51.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:51 smithi189 ceph-mon[32912]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-13T13:38:51.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:51.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:51 smithi189 ceph-mon[32912]: osdmap e448: 8 total, 8 up, 8 in 2024-09-13T13:38:51.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[30106]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-13T13:38:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[30106]: osdmap e448: 8 total, 8 up, 8 in 2024-09-13T13:38:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:51.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:51.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[40255]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-13T13:38:51.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_tier","val": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:38:51.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[40255]: osdmap e448: 8 total, 8 up, 8 in 2024-09-13T13:38:51.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:51.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:38:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:53 smithi189 ceph-mon[32912]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-13T13:38:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:53 smithi189 ceph-mon[32912]: osdmap e449: 8 total, 8 up, 8 in 2024-09-13T13:38:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[30106]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-13T13:38:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[30106]: osdmap e449: 8 total, 8 up, 8 in 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[40255]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[40255]: osdmap e449: 8 total, 8 up, 8 in 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:38:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:54 smithi189 ceph-mon[32912]: osdmap e450: 8 total, 8 up, 8 in 2024-09-13T13:38:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:54 smithi043 ceph-mon[30106]: osdmap e450: 8 total, 8 up, 8 in 2024-09-13T13:38:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:38:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:54 smithi043 ceph-mon[40255]: osdmap e450: 8 total, 8 up, 8 in 2024-09-13T13:38:55.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:38:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:38:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:55 smithi189 ceph-mon[32912]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:55 smithi189 ceph-mon[32912]: osdmap e451: 8 total, 8 up, 8 in 2024-09-13T13:38:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:38:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:38:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:38:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:55 smithi043 ceph-mon[40255]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:55 smithi043 ceph-mon[40255]: osdmap e451: 8 total, 8 up, 8 in 2024-09-13T13:38:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:55 smithi043 ceph-mon[30106]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:38:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:55 smithi043 ceph-mon[30106]: osdmap e451: 8 total, 8 up, 8 in 2024-09-13T13:38:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:56 smithi189 ceph-mon[32912]: osdmap e452: 8 total, 8 up, 8 in 2024-09-13T13:38:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-85"} : dispatch 2024-09-13T13:38:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-85"} : dispatch 2024-09-13T13:38:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[30106]: osdmap e452: 8 total, 8 up, 8 in 2024-09-13T13:38:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-85"} : dispatch 2024-09-13T13:38:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-85"} : dispatch 2024-09-13T13:38:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[40255]: osdmap e452: 8 total, 8 up, 8 in 2024-09-13T13:38:56.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:56.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:38:56.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-85"} : dispatch 2024-09-13T13:38:56.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-85"} : dispatch 2024-09-13T13:38:56.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:38:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:57 smithi189 ceph-mon[32912]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-13T13:38:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:57 smithi189 ceph-mon[32912]: osdmap e453: 8 total, 8 up, 8 in 2024-09-13T13:38:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:57 smithi043 ceph-mon[30106]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-13T13:38:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:57 smithi043 ceph-mon[30106]: osdmap e453: 8 total, 8 up, 8 in 2024-09-13T13:38:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:57 smithi043 ceph-mon[40255]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-13T13:38:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:57 smithi043 ceph-mon[40255]: osdmap e453: 8 total, 8 up, 8 in 2024-09-13T13:38:58.576 INFO:tasks.workunit.client.0.smithi043.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18076 ms) 2024-09-13T13:38:58.576 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-13T13:38:58.576 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22160 ms) 2024-09-13T13:38:58.576 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-13T13:38:58.576 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3032 ms) 2024-09-13T13:38:58.576 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-13T13:38:58.576 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3012 ms) 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3032 ms) 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3019 ms) 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4027 ms) 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39917 ms) 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16982 ms) 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4028 ms) 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3014 ms) 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25087 ms) 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-13T13:38:58.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14208 ms) 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5039 ms) 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4029 ms) 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24621 ms) 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-13T13:38:58.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10065 ms) 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9047 ms) 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9038 ms) 2024-09-13T13:38:58.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-13T13:38:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:58 smithi189 ceph-mon[32912]: osdmap e454: 8 total, 8 up, 8 in 2024-09-13T13:38:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:58 smithi043 ceph-mon[40255]: osdmap e454: 8 total, 8 up, 8 in 2024-09-13T13:38:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:58 smithi043 ceph-mon[30106]: osdmap e454: 8 total, 8 up, 8 in 2024-09-13T13:38:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:38:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:59 smithi189 ceph-mon[32912]: pgmap v676: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:38:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:59 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:59 smithi189 ceph-mon[32912]: osdmap e455: 8 total, 8 up, 8 in 2024-09-13T13:38:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:38:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[30106]: pgmap v676: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[30106]: osdmap e455: 8 total, 8 up, 8 in 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[40255]: pgmap v676: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[40255]: osdmap e455: 8 total, 8 up, 8 in 2024-09-13T13:38:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:38:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:00 smithi189 ceph-mon[32912]: osdmap e456: 8 total, 8 up, 8 in 2024-09-13T13:39:00.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:00.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:00 smithi043 ceph-mon[30106]: osdmap e456: 8 total, 8 up, 8 in 2024-09-13T13:39:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:00 smithi043 ceph-mon[40255]: osdmap e456: 8 total, 8 up, 8 in 2024-09-13T13:39:00.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:00.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:01 smithi189 ceph-mon[32912]: pgmap v679: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-13T13:39:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:39:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:01 smithi189 ceph-mon[32912]: osdmap e457: 8 total, 8 up, 8 in 2024-09-13T13:39:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"} : dispatch 2024-09-13T13:39:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"} : dispatch 2024-09-13T13:39:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[30106]: pgmap v679: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[30106]: osdmap e457: 8 total, 8 up, 8 in 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"} : dispatch 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"} : dispatch 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[40255]: pgmap v679: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[40255]: osdmap e457: 8 total, 8 up, 8 in 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"} : dispatch 2024-09-13T13:39:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"} : dispatch 2024-09-13T13:39:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"}]': finished 2024-09-13T13:39:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:02 smithi189 ceph-mon[32912]: osdmap e458: 8 total, 8 up, 8 in 2024-09-13T13:39:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:39:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:39:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"}]': finished 2024-09-13T13:39:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[30106]: osdmap e458: 8 total, 8 up, 8 in 2024-09-13T13:39:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:39:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:39:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_tier","val": "test-rados-api-smithi043-76627-89-test-flush"}]': finished 2024-09-13T13:39:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[40255]: osdmap e458: 8 total, 8 up, 8 in 2024-09-13T13:39:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:39:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:39:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[30106]: pgmap v682: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-13T13:39:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:39:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[30106]: osdmap e459: 8 total, 8 up, 8 in 2024-09-13T13:39:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:39:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:39:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[40255]: pgmap v682: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-13T13:39:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:39:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[40255]: osdmap e459: 8 total, 8 up, 8 in 2024-09-13T13:39:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:39:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:39:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:03 smithi189 ceph-mon[32912]: pgmap v682: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-13T13:39:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:39:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:03 smithi189 ceph-mon[32912]: osdmap e459: 8 total, 8 up, 8 in 2024-09-13T13:39:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:39:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:39:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:39:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:04 smithi189 ceph-mon[32912]: osdmap e460: 8 total, 8 up, 8 in 2024-09-13T13:39:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:39:04.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:04 smithi043 ceph-mon[30106]: osdmap e460: 8 total, 8 up, 8 in 2024-09-13T13:39:04.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:39:04.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:04 smithi043 ceph-mon[40255]: osdmap e460: 8 total, 8 up, 8 in 2024-09-13T13:39:05.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:39:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-mon[32912]: pgmap v685: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-mon[32912]: osdmap e461: 8 total, 8 up, 8 in 2024-09-13T13:39:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-87"} : dispatch 2024-09-13T13:39:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-87"} : dispatch 2024-09-13T13:39:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:39:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[30106]: pgmap v685: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[30106]: osdmap e461: 8 total, 8 up, 8 in 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-87"} : dispatch 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-87"} : dispatch 2024-09-13T13:39:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[40255]: pgmap v685: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[40255]: osdmap e461: 8 total, 8 up, 8 in 2024-09-13T13:39:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-87"} : dispatch 2024-09-13T13:39:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-87"} : dispatch 2024-09-13T13:39:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:06 smithi189 ceph-mon[32912]: osdmap e462: 8 total, 8 up, 8 in 2024-09-13T13:39:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:06 smithi043 ceph-mon[30106]: osdmap e462: 8 total, 8 up, 8 in 2024-09-13T13:39:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:06 smithi043 ceph-mon[40255]: osdmap e462: 8 total, 8 up, 8 in 2024-09-13T13:39:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:07 smithi189 ceph-mon[32912]: pgmap v688: 196 pgs: 196 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:39:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:07 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:07 smithi189 ceph-mon[32912]: osdmap e463: 8 total, 8 up, 8 in 2024-09-13T13:39:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[30106]: pgmap v688: 196 pgs: 196 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[30106]: osdmap e463: 8 total, 8 up, 8 in 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[40255]: pgmap v688: 196 pgs: 196 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[40255]: osdmap e463: 8 total, 8 up, 8 in 2024-09-13T13:39:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:07.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:08.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:08 smithi189 ceph-mon[32912]: osdmap e464: 8 total, 8 up, 8 in 2024-09-13T13:39:08.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:08.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:08.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[30106]: osdmap e464: 8 total, 8 up, 8 in 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[40255]: osdmap e464: 8 total, 8 up, 8 in 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:39:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:09 smithi189 ceph-mon[32912]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:39:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:39:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:09 smithi189 ceph-mon[32912]: osdmap e465: 8 total, 8 up, 8 in 2024-09-13T13:39:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:09 smithi043 ceph-mon[30106]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:39:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:39:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:09 smithi043 ceph-mon[30106]: osdmap e465: 8 total, 8 up, 8 in 2024-09-13T13:39:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:09 smithi043 ceph-mon[40255]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:39:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:39:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:09 smithi043 ceph-mon[40255]: osdmap e465: 8 total, 8 up, 8 in 2024-09-13T13:39:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:10 smithi189 ceph-mon[32912]: osdmap e466: 8 total, 8 up, 8 in 2024-09-13T13:39:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:10 smithi043 ceph-mon[30106]: osdmap e466: 8 total, 8 up, 8 in 2024-09-13T13:39:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:10 smithi043 ceph-mon[40255]: osdmap e466: 8 total, 8 up, 8 in 2024-09-13T13:39:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[30106]: pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 3.0 KiB/s wr, 104 op/s 2024-09-13T13:39:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[30106]: osdmap e467: 8 total, 8 up, 8 in 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-90"} : dispatch 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-90"} : dispatch 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[40255]: pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 3.0 KiB/s wr, 104 op/s 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[40255]: osdmap e467: 8 total, 8 up, 8 in 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:11.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-90"} : dispatch 2024-09-13T13:39:11.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-90"} : dispatch 2024-09-13T13:39:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:11 smithi189 ceph-mon[32912]: pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 3.0 KiB/s wr, 104 op/s 2024-09-13T13:39:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:11 smithi189 ceph-mon[32912]: osdmap e467: 8 total, 8 up, 8 in 2024-09-13T13:39:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-90"} : dispatch 2024-09-13T13:39:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-90"} : dispatch 2024-09-13T13:39:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:12 smithi043 ceph-mon[30106]: osdmap e468: 8 total, 8 up, 8 in 2024-09-13T13:39:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:12 smithi043 ceph-mon[40255]: osdmap e468: 8 total, 8 up, 8 in 2024-09-13T13:39:13.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:12 smithi189 ceph-mon[32912]: osdmap e468: 8 total, 8 up, 8 in 2024-09-13T13:39:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[40255]: pgmap v697: 196 pgs: 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 1023 B/s wr, 100 op/s 2024-09-13T13:39:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[40255]: osdmap e469: 8 total, 8 up, 8 in 2024-09-13T13:39:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[30106]: pgmap v697: 196 pgs: 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 1023 B/s wr, 100 op/s 2024-09-13T13:39:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[30106]: osdmap e469: 8 total, 8 up, 8 in 2024-09-13T13:39:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:14.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:13 smithi189 ceph-mon[32912]: pgmap v697: 196 pgs: 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 1023 B/s wr, 100 op/s 2024-09-13T13:39:14.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:13 smithi189 ceph-mon[32912]: osdmap e469: 8 total, 8 up, 8 in 2024-09-13T13:39:14.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:14.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:14.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:14.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:14 smithi043 ceph-mon[30106]: osdmap e470: 8 total, 8 up, 8 in 2024-09-13T13:39:14.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:14 smithi043 ceph-mon[40255]: osdmap e470: 8 total, 8 up, 8 in 2024-09-13T13:39:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:14 smithi189 ceph-mon[32912]: osdmap e470: 8 total, 8 up, 8 in 2024-09-13T13:39:15.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:15 smithi043 ceph-mon[30106]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:15 smithi043 ceph-mon[30106]: osdmap e471: 8 total, 8 up, 8 in 2024-09-13T13:39:15.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:39:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:39:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:39:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:15 smithi043 ceph-mon[40255]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:15 smithi043 ceph-mon[40255]: osdmap e471: 8 total, 8 up, 8 in 2024-09-13T13:39:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:15 smithi189 ceph-mon[32912]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:15 smithi189 ceph-mon[32912]: osdmap e471: 8 total, 8 up, 8 in 2024-09-13T13:39:16.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:39:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:39:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[30106]: osdmap e472: 8 total, 8 up, 8 in 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-92"} : dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-92"} : dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[40255]: osdmap e472: 8 total, 8 up, 8 in 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-92"} : dispatch 2024-09-13T13:39:16.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-92"} : dispatch 2024-09-13T13:39:17.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:17.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:16 smithi189 ceph-mon[32912]: osdmap e472: 8 total, 8 up, 8 in 2024-09-13T13:39:17.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:17.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:17.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-92"} : dispatch 2024-09-13T13:39:17.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-92"} : dispatch 2024-09-13T13:39:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:17 smithi043 ceph-mon[30106]: pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 1.7 KiB/s wr, 83 op/s 2024-09-13T13:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:17 smithi043 ceph-mon[30106]: osdmap e473: 8 total, 8 up, 8 in 2024-09-13T13:39:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:17 smithi043 ceph-mon[40255]: pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 1.7 KiB/s wr, 83 op/s 2024-09-13T13:39:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:17 smithi043 ceph-mon[40255]: osdmap e473: 8 total, 8 up, 8 in 2024-09-13T13:39:18.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:17 smithi189 ceph-mon[32912]: pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 1.7 KiB/s wr, 83 op/s 2024-09-13T13:39:18.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:17 smithi189 ceph-mon[32912]: osdmap e473: 8 total, 8 up, 8 in 2024-09-13T13:39:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:18 smithi043 ceph-mon[40255]: osdmap e474: 8 total, 8 up, 8 in 2024-09-13T13:39:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:18 smithi043 ceph-mon[30106]: osdmap e474: 8 total, 8 up, 8 in 2024-09-13T13:39:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:19.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:18 smithi189 ceph-mon[32912]: osdmap e474: 8 total, 8 up, 8 in 2024-09-13T13:39:19.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:19.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[40255]: pgmap v706: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 1023 B/s wr, 82 op/s 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[40255]: osdmap e475: 8 total, 8 up, 8 in 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[30106]: pgmap v706: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 1023 B/s wr, 82 op/s 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[30106]: osdmap e475: 8 total, 8 up, 8 in 2024-09-13T13:39:19.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:19 smithi189 ceph-mon[32912]: pgmap v706: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 1023 B/s wr, 82 op/s 2024-09-13T13:39:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:19 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:19 smithi189 ceph-mon[32912]: osdmap e475: 8 total, 8 up, 8 in 2024-09-13T13:39:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:20 smithi043 ceph-mon[40255]: osdmap e476: 8 total, 8 up, 8 in 2024-09-13T13:39:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:20 smithi043 ceph-mon[30106]: osdmap e476: 8 total, 8 up, 8 in 2024-09-13T13:39:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:21.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:20 smithi189 ceph-mon[32912]: osdmap e476: 8 total, 8 up, 8 in 2024-09-13T13:39:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:21 smithi043 ceph-mon[40255]: pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:39:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:21 smithi043 ceph-mon[40255]: osdmap e477: 8 total, 8 up, 8 in 2024-09-13T13:39:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:21 smithi043 ceph-mon[30106]: pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:39:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:21 smithi043 ceph-mon[30106]: osdmap e477: 8 total, 8 up, 8 in 2024-09-13T13:39:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:21 smithi189 ceph-mon[32912]: pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:39:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:21 smithi189 ceph-mon[32912]: osdmap e477: 8 total, 8 up, 8 in 2024-09-13T13:39:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:23 smithi043 ceph-mon[40255]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:39:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:23 smithi043 ceph-mon[30106]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:39:24.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:23 smithi189 ceph-mon[32912]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:39:25.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:25 smithi043 ceph-mon[40255]: pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-13T13:39:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:25 smithi043 ceph-mon[30106]: pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-13T13:39:25.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:39:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:39:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:39:26.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:39:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:39:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:25 smithi189 ceph-mon[32912]: pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-13T13:39:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:27 smithi043 ceph-mon[30106]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-13T13:39:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:27 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:27 smithi043 ceph-mon[40255]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-13T13:39:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:27 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:27 smithi189 ceph-mon[32912]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-13T13:39:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:27 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:29 smithi043 ceph-mon[30106]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-13T13:39:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:29 smithi043 ceph-mon[40255]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-13T13:39:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:29 smithi189 ceph-mon[32912]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-13T13:39:32.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:31 smithi189 ceph-mon[32912]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:31 smithi043 ceph-mon[30106]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:31 smithi043 ceph-mon[40255]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:33.143 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:32 smithi043 ceph-mon[40255]: osdmap e478: 8 total, 8 up, 8 in 2024-09-13T13:39:33.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:32 smithi043 ceph-mon[30106]: osdmap e478: 8 total, 8 up, 8 in 2024-09-13T13:39:33.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:32 smithi189 ceph-mon[32912]: osdmap e478: 8 total, 8 up, 8 in 2024-09-13T13:39:34.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:33 smithi189 ceph-mon[32912]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:34.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:33 smithi043 ceph-mon[30106]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:34.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:33 smithi043 ceph-mon[40255]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:35 smithi043 ceph-mon[40255]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:35 smithi043 ceph-mon[30106]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:35.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:39:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:39:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:39:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:35 smithi189 ceph-mon[32912]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-13T13:39:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:36.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:39:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:39:37.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:37 smithi189 ceph-mon[32912]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:37 smithi189 ceph-mon[32912]: osdmap e479: 8 total, 8 up, 8 in 2024-09-13T13:39:38.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:37 smithi043 ceph-mon[30106]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:37 smithi043 ceph-mon[30106]: osdmap e479: 8 total, 8 up, 8 in 2024-09-13T13:39:38.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:37 smithi043 ceph-mon[40255]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:38.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:37 smithi043 ceph-mon[40255]: osdmap e479: 8 total, 8 up, 8 in 2024-09-13T13:39:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:39 smithi189 ceph-mon[32912]: pgmap v721: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:40.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:39 smithi043 ceph-mon[30106]: pgmap v721: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:39 smithi043 ceph-mon[40255]: pgmap v721: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:41 smithi189 ceph-mon[32912]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:42.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:41 smithi043 ceph-mon[30106]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:41 smithi043 ceph-mon[40255]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-94"} : dispatch 2024-09-13T13:39:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-94"} : dispatch 2024-09-13T13:39:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-94"} : dispatch 2024-09-13T13:39:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-94"} : dispatch 2024-09-13T13:39:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-94"} : dispatch 2024-09-13T13:39:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-94"} : dispatch 2024-09-13T13:39:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:43 smithi189 ceph-mon[32912]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:43 smithi189 ceph-mon[32912]: osdmap e480: 8 total, 8 up, 8 in 2024-09-13T13:39:44.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:43 smithi043 ceph-mon[40255]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:44.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:43 smithi043 ceph-mon[40255]: osdmap e480: 8 total, 8 up, 8 in 2024-09-13T13:39:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:43 smithi043 ceph-mon[30106]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:39:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:43 smithi043 ceph-mon[30106]: osdmap e480: 8 total, 8 up, 8 in 2024-09-13T13:39:45.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:44 smithi189 ceph-mon[32912]: osdmap e481: 8 total, 8 up, 8 in 2024-09-13T13:39:45.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:45.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:45.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:44 smithi043 ceph-mon[40255]: osdmap e481: 8 total, 8 up, 8 in 2024-09-13T13:39:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:44 smithi043 ceph-mon[30106]: osdmap e481: 8 total, 8 up, 8 in 2024-09-13T13:39:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:39:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:39:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[40255]: pgmap v726: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[40255]: osdmap e482: 8 total, 8 up, 8 in 2024-09-13T13:39:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[30106]: pgmap v726: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[30106]: osdmap e482: 8 total, 8 up, 8 in 2024-09-13T13:39:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:46.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:39:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:39:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:45 smithi189 ceph-mon[32912]: pgmap v726: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:46.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:45 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:46.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:39:46.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:45 smithi189 ceph-mon[32912]: osdmap e482: 8 total, 8 up, 8 in 2024-09-13T13:39:46.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:39:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:46 smithi189 ceph-mon[32912]: osdmap e483: 8 total, 8 up, 8 in 2024-09-13T13:39:47.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:47.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:46 smithi043 ceph-mon[30106]: osdmap e483: 8 total, 8 up, 8 in 2024-09-13T13:39:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:46 smithi043 ceph-mon[40255]: osdmap e483: 8 total, 8 up, 8 in 2024-09-13T13:39:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:47 smithi189 ceph-mon[32912]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:48.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:47 smithi043 ceph-mon[30106]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:47 smithi043 ceph-mon[40255]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:39:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:49 smithi189 ceph-mon[32912]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 916 B/s rd, 916 B/s wr, 1 op/s 2024-09-13T13:39:50.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:49 smithi043 ceph-mon[30106]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 916 B/s rd, 916 B/s wr, 1 op/s 2024-09-13T13:39:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:49 smithi043 ceph-mon[40255]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 916 B/s rd, 916 B/s wr, 1 op/s 2024-09-13T13:39:51.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:51.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:39:52.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 933 B/s wr, 3 op/s 2024-09-13T13:39:52.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:39:52.150 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 933 B/s wr, 3 op/s 2024-09-13T13:39:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:39:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 933 B/s wr, 3 op/s 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:52.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:39:52.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:39:52.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:39:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:52 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-13T13:39:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:52 smithi189 ceph-mon[32912]: Adjusting osd_memory_target on smithi189 to 3248M 2024-09-13T13:39:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:52 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:52 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-13T13:39:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:52 smithi043 ceph-mon[30106]: Adjusting osd_memory_target on smithi189 to 3248M 2024-09-13T13:39:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:52 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:52 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-13T13:39:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:52 smithi043 ceph-mon[40255]: Adjusting osd_memory_target on smithi189 to 3248M 2024-09-13T13:39:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:52 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:39:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:53 smithi189 ceph-mon[32912]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:39:54.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:53 smithi043 ceph-mon[30106]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:39:54.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:53 smithi043 ceph-mon[40255]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-13T13:39:55.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:55 smithi043 ceph-mon[40255]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 641 B/s wr, 2 op/s 2024-09-13T13:39:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:55 smithi043 ceph-mon[30106]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 641 B/s wr, 2 op/s 2024-09-13T13:39:55.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:39:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:39:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:39:55.992 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:39:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2[56012]: 2024-09-13T13:39:55.822+0000 7f6a106f7640 -1 snap_mapper.add_oid found existing snaps mapped on 108:f20c9eb9:test-rados-api-smithi043-76627-97::foo:21, removing 2024-09-13T13:39:56.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:55 smithi189 ceph-mon[32912]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 641 B/s wr, 2 op/s 2024-09-13T13:39:56.148 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:39:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:39:55.822+0000 7f6db0c94640 -1 snap_mapper.add_oid found existing snaps mapped on 108:f20c9eb9:test-rados-api-smithi043-76627-97::foo:21, removing 2024-09-13T13:39:56.149 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:39:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:39:55.823+0000 7fcada926640 -1 snap_mapper.add_oid found existing snaps mapped on 108:f20c9eb9:test-rados-api-smithi043-76627-97::foo:21, removing 2024-09-13T13:39:56.149 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:39:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:39:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:57.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-96"} : dispatch 2024-09-13T13:39:57.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-96"} : dispatch 2024-09-13T13:39:57.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-96"} : dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-96"} : dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-96"} : dispatch 2024-09-13T13:39:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-96"} : dispatch 2024-09-13T13:39:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:57 smithi189 ceph-mon[32912]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 96 B/s wr, 1 op/s 2024-09-13T13:39:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:57 smithi189 ceph-mon[32912]: osdmap e484: 8 total, 8 up, 8 in 2024-09-13T13:39:58.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:57 smithi043 ceph-mon[40255]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 96 B/s wr, 1 op/s 2024-09-13T13:39:58.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:57 smithi043 ceph-mon[40255]: osdmap e484: 8 total, 8 up, 8 in 2024-09-13T13:39:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:57 smithi043 ceph-mon[30106]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 96 B/s wr, 1 op/s 2024-09-13T13:39:58.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:57 smithi043 ceph-mon[30106]: osdmap e484: 8 total, 8 up, 8 in 2024-09-13T13:39:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:58 smithi189 ceph-mon[32912]: osdmap e485: 8 total, 8 up, 8 in 2024-09-13T13:39:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:59.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:58 smithi043 ceph-mon[30106]: osdmap e485: 8 total, 8 up, 8 in 2024-09-13T13:39:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:58 smithi043 ceph-mon[40255]: osdmap e485: 8 total, 8 up, 8 in 2024-09-13T13:39:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:39:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:00.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:59 smithi189 ceph-mon[32912]: pgmap v737: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:40:00.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:59 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:00.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:59 smithi189 ceph-mon[32912]: osdmap e486: 8 total, 8 up, 8 in 2024-09-13T13:40:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:00.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:39:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:00.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[30106]: pgmap v737: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[30106]: osdmap e486: 8 total, 8 up, 8 in 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[40255]: pgmap v737: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[40255]: osdmap e486: 8 total, 8 up, 8 in 2024-09-13T13:40:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:00.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:39:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:01.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: osdmap e487: 8 total, 8 up, 8 in 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"} : dispatch 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"} : dispatch 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: application not enabled on pool 'test-rados-api-smithi043-76627-98' 2024-09-13T13:40:01.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:00 smithi189 ceph-mon[32912]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: osdmap e487: 8 total, 8 up, 8 in 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"} : dispatch 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"} : dispatch 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: application not enabled on pool 'test-rados-api-smithi043-76627-98' 2024-09-13T13:40:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[30106]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: osdmap e487: 8 total, 8 up, 8 in 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"} : dispatch 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"} : dispatch 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: application not enabled on pool 'test-rados-api-smithi043-76627-98' 2024-09-13T13:40:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:00 smithi043 ceph-mon[40255]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-13T13:40:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:01 smithi189 ceph-mon[32912]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-13T13:40:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:01 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"}]': finished 2024-09-13T13:40:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:01 smithi189 ceph-mon[32912]: osdmap e488: 8 total, 8 up, 8 in 2024-09-13T13:40:02.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:02.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[30106]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"}]': finished 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[30106]: osdmap e488: 8 total, 8 up, 8 in 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[40255]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-98", "mode": "writeback"}]': finished 2024-09-13T13:40:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[40255]: osdmap e488: 8 total, 8 up, 8 in 2024-09-13T13:40:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:02.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:03.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"}]': finished 2024-09-13T13:40:03.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[30106]: osdmap e489: 8 total, 8 up, 8 in 2024-09-13T13:40:03.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:40:03.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:40:03.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[30106]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:40:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"}]': finished 2024-09-13T13:40:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[40255]: osdmap e489: 8 total, 8 up, 8 in 2024-09-13T13:40:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:40:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:40:03.144 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:02 smithi043 ceph-mon[40255]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:40:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-98"}]': finished 2024-09-13T13:40:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:02 smithi189 ceph-mon[32912]: osdmap e489: 8 total, 8 up, 8 in 2024-09-13T13:40:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:40:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:40:03.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:02 smithi189 ceph-mon[32912]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:40:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:03 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:40:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:03 smithi189 ceph-mon[32912]: osdmap e490: 8 total, 8 up, 8 in 2024-09-13T13:40:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-13T13:40:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-13T13:40:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[40255]: osdmap e490: 8 total, 8 up, 8 in 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[30106]: osdmap e490: 8 total, 8 up, 8 in 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-13T13:40:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-13T13:40:05.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"}]': finished 2024-09-13T13:40:05.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:04 smithi189 ceph-mon[32912]: osdmap e491: 8 total, 8 up, 8 in 2024-09-13T13:40:05.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:05.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:05.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:04 smithi189 ceph-mon[32912]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"}]': finished 2024-09-13T13:40:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[30106]: osdmap e491: 8 total, 8 up, 8 in 2024-09-13T13:40:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[30106]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_count","val": "1"}]': finished 2024-09-13T13:40:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[40255]: osdmap e491: 8 total, 8 up, 8 in 2024-09-13T13:40:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:05.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:04 smithi043 ceph-mon[40255]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:40:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[40255]: osdmap e492: 8 total, 8 up, 8 in 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[30106]: osdmap e492: 8 total, 8 up, 8 in 2024-09-13T13:40:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-13T13:40:05.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-13T13:40:05.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:05.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:05 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:06.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:40:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:40:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:05 smithi189 ceph-mon[32912]: osdmap e492: 8 total, 8 up, 8 in 2024-09-13T13:40:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-13T13:40:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-13T13:40:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:05 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"}]': finished 2024-09-13T13:40:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:06 smithi189 ceph-mon[32912]: osdmap e493: 8 total, 8 up, 8 in 2024-09-13T13:40:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:06 smithi189 ceph-mon[32912]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:07.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"}]': finished 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[30106]: osdmap e493: 8 total, 8 up, 8 in 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[30106]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-98","var": "target_max_objects","val": "250"}]': finished 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[40255]: osdmap e493: 8 total, 8 up, 8 in 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:06 smithi043 ceph-mon[40255]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:08.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[30106]: osdmap e494: 8 total, 8 up, 8 in 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[40255]: osdmap e494: 8 total, 8 up, 8 in 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:08.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:07 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:40:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:07 smithi189 ceph-mon[32912]: osdmap e494: 8 total, 8 up, 8 in 2024-09-13T13:40:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"} : dispatch 2024-09-13T13:40:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:07 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"}]': finished 2024-09-13T13:40:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:08 smithi043 ceph-mon[30106]: osdmap e495: 8 total, 8 up, 8 in 2024-09-13T13:40:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:08 smithi043 ceph-mon[30106]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"}]': finished 2024-09-13T13:40:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:08 smithi043 ceph-mon[40255]: osdmap e495: 8 total, 8 up, 8 in 2024-09-13T13:40:09.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:08 smithi043 ceph-mon[40255]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-98"}]': finished 2024-09-13T13:40:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:08 smithi189 ceph-mon[32912]: osdmap e495: 8 total, 8 up, 8 in 2024-09-13T13:40:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:08 smithi189 ceph-mon[32912]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:09 smithi189 ceph-mon[32912]: osdmap e496: 8 total, 8 up, 8 in 2024-09-13T13:40:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:40:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=cleanup t=2024-09-13T13:40:09.687084188Z level=info msg="Completed cleanup jobs" duration=12.391444ms 2024-09-13T13:40:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:40:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=plugins.update.checker t=2024-09-13T13:40:09.8141927Z level=info msg="Update check succeeded" duration=59.736557ms 2024-09-13T13:40:10.149 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:40:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=grafana.update.checker t=2024-09-13T13:40:09.825843593Z level=info msg="Update check succeeded" duration=71.33428ms 2024-09-13T13:40:10.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:09 smithi043 ceph-mon[30106]: osdmap e496: 8 total, 8 up, 8 in 2024-09-13T13:40:10.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:09 smithi043 ceph-mon[40255]: osdmap e496: 8 total, 8 up, 8 in 2024-09-13T13:40:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[30106]: osdmap e497: 8 total, 8 up, 8 in 2024-09-13T13:40:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[30106]: pgmap v755: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[40255]: osdmap e497: 8 total, 8 up, 8 in 2024-09-13T13:40:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:10 smithi043 ceph-mon[40255]: pgmap v755: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:10 smithi189 ceph-mon[32912]: osdmap e497: 8 total, 8 up, 8 in 2024-09-13T13:40:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:10 smithi189 ceph-mon[32912]: pgmap v755: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[30106]: osdmap e498: 8 total, 8 up, 8 in 2024-09-13T13:40:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[40255]: osdmap e498: 8 total, 8 up, 8 in 2024-09-13T13:40:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:12.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:11 smithi189 ceph-mon[32912]: osdmap e498: 8 total, 8 up, 8 in 2024-09-13T13:40:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:12.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[30106]: osdmap e499: 8 total, 8 up, 8 in 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[30106]: pgmap v758: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[40255]: osdmap e499: 8 total, 8 up, 8 in 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:13.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:13.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:12 smithi043 ceph-mon[40255]: pgmap v758: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:40:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:12 smithi189 ceph-mon[32912]: osdmap e499: 8 total, 8 up, 8 in 2024-09-13T13:40:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:12 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:13.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:12 smithi189 ceph-mon[32912]: pgmap v758: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"}]': finished 2024-09-13T13:40:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[30106]: osdmap e500: 8 total, 8 up, 8 in 2024-09-13T13:40:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"} : dispatch 2024-09-13T13:40:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"} : dispatch 2024-09-13T13:40:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"}]': finished 2024-09-13T13:40:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[40255]: osdmap e500: 8 total, 8 up, 8 in 2024-09-13T13:40:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"} : dispatch 2024-09-13T13:40:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"} : dispatch 2024-09-13T13:40:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-6", "overlaypool": "test-rados-api-smithi043-76627-100"}]': finished 2024-09-13T13:40:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:13 smithi189 ceph-mon[32912]: osdmap e500: 8 total, 8 up, 8 in 2024-09-13T13:40:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"} : dispatch 2024-09-13T13:40:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"} : dispatch 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"}]': finished 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[30106]: osdmap e501: 8 total, 8 up, 8 in 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[30106]: pgmap v761: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:15.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"}]': finished 2024-09-13T13:40:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[40255]: osdmap e501: 8 total, 8 up, 8 in 2024-09-13T13:40:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[40255]: pgmap v761: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:15.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:40:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:14 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-100", "mode": "writeback"}]': finished 2024-09-13T13:40:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:14 smithi189 ceph-mon[32912]: osdmap e501: 8 total, 8 up, 8 in 2024-09-13T13:40:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:14 smithi189 ceph-mon[32912]: pgmap v761: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:40:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[30106]: osdmap e502: 8 total, 8 up, 8 in 2024-09-13T13:40:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:40:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[40255]: osdmap e502: 8 total, 8 up, 8 in 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:15.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:15 smithi189 ceph-mon[32912]: osdmap e502: 8 total, 8 up, 8 in 2024-09-13T13:40:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:15 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:16.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:16.149 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:40:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:40:17.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[30106]: osdmap e503: 8 total, 8 up, 8 in 2024-09-13T13:40:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:17.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[30106]: pgmap v764: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[40255]: osdmap e503: 8 total, 8 up, 8 in 2024-09-13T13:40:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:17.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:16 smithi043 ceph-mon[40255]: pgmap v764: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:16 smithi189 ceph-mon[32912]: osdmap e503: 8 total, 8 up, 8 in 2024-09-13T13:40:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:16 smithi189 ceph-mon[32912]: pgmap v764: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:18.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[30106]: osdmap e504: 8 total, 8 up, 8 in 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[40255]: osdmap e504: 8 total, 8 up, 8 in 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-13T13:40:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-13T13:40:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:17 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:40:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:17 smithi189 ceph-mon[32912]: osdmap e504: 8 total, 8 up, 8 in 2024-09-13T13:40:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:17 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-13T13:40:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-13T13:40:19.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[30106]: osdmap e505: 8 total, 8 up, 8 in 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[30106]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[40255]: osdmap e505: 8 total, 8 up, 8 in 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:18 smithi043 ceph-mon[40255]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-13T13:40:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:18 smithi189 ceph-mon[32912]: osdmap e505: 8 total, 8 up, 8 in 2024-09-13T13:40:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:18 smithi189 ceph-mon[32912]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:40:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[40255]: osdmap e506: 8 total, 8 up, 8 in 2024-09-13T13:40:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:20.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:40:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[30106]: osdmap e506: 8 total, 8 up, 8 in 2024-09-13T13:40:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"}]': finished 2024-09-13T13:40:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:19 smithi189 ceph-mon[32912]: osdmap e506: 8 total, 8 up, 8 in 2024-09-13T13:40:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"} : dispatch 2024-09-13T13:40:21.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"}]': finished 2024-09-13T13:40:21.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[40255]: osdmap e507: 8 total, 8 up, 8 in 2024-09-13T13:40:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:21.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[40255]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:40:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"}]': finished 2024-09-13T13:40:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[30106]: osdmap e507: 8 total, 8 up, 8 in 2024-09-13T13:40:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:20 smithi043 ceph-mon[30106]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:40:21.299 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-100"}]': finished 2024-09-13T13:40:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:20 smithi189 ceph-mon[32912]: osdmap e507: 8 total, 8 up, 8 in 2024-09-13T13:40:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:21.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:20 smithi189 ceph-mon[32912]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:40:22.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:21 smithi043 ceph-mon[30106]: osdmap e508: 8 total, 8 up, 8 in 2024-09-13T13:40:22.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:21 smithi043 ceph-mon[40255]: osdmap e508: 8 total, 8 up, 8 in 2024-09-13T13:40:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:21 smithi189 ceph-mon[32912]: osdmap e508: 8 total, 8 up, 8 in 2024-09-13T13:40:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:23 smithi189 ceph-mon[32912]: osdmap e509: 8 total, 8 up, 8 in 2024-09-13T13:40:23.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:23.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:23.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:23 smithi189 ceph-mon[32912]: pgmap v773: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:23.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[30106]: osdmap e509: 8 total, 8 up, 8 in 2024-09-13T13:40:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[30106]: pgmap v773: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[40255]: osdmap e509: 8 total, 8 up, 8 in 2024-09-13T13:40:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:22 smithi043 ceph-mon[40255]: pgmap v773: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:24 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:24 smithi189 ceph-mon[32912]: osdmap e510: 8 total, 8 up, 8 in 2024-09-13T13:40:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:40:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:40:24.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[30106]: osdmap e510: 8 total, 8 up, 8 in 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:24.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[40255]: osdmap e510: 8 total, 8 up, 8 in 2024-09-13T13:40:24.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-13T13:40:24.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:40:24.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-13T13:40:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:40:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:25 smithi189 ceph-mon[32912]: osdmap e511: 8 total, 8 up, 8 in 2024-09-13T13:40:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:40:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:40:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:25 smithi189 ceph-mon[32912]: pgmap v776: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:40:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[30106]: osdmap e511: 8 total, 8 up, 8 in 2024-09-13T13:40:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:40:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:40:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[30106]: pgmap v776: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-13T13:40:25.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[40255]: osdmap e511: 8 total, 8 up, 8 in 2024-09-13T13:40:25.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:40:25.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-13T13:40:25.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-mon[40255]: pgmap v776: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:40:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:40:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:40:26.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:40:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:40:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:40:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:26 smithi189 ceph-mon[32912]: osdmap e512: 8 total, 8 up, 8 in 2024-09-13T13:40:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:40:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:40:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:40:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[30106]: osdmap e512: 8 total, 8 up, 8 in 2024-09-13T13:40:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:40:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:40:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:26.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-13T13:40:26.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[40255]: osdmap e512: 8 total, 8 up, 8 in 2024-09-13T13:40:26.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:40:26.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-13T13:40:26.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:40:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:27 smithi189 ceph-mon[32912]: osdmap e513: 8 total, 8 up, 8 in 2024-09-13T13:40:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-102"} : dispatch 2024-09-13T13:40:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-102"} : dispatch 2024-09-13T13:40:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:27 smithi189 ceph-mon[32912]: pgmap v779: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[30106]: osdmap e513: 8 total, 8 up, 8 in 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-102"} : dispatch 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-102"} : dispatch 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[30106]: pgmap v779: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-13T13:40:27.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[40255]: osdmap e513: 8 total, 8 up, 8 in 2024-09-13T13:40:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-6"} : dispatch 2024-09-13T13:40:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/283219024' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-102"} : dispatch 2024-09-13T13:40:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-6", "tierpool": "test-rados-api-smithi043-76627-102"} : dispatch 2024-09-13T13:40:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:27 smithi043 ceph-mon[40255]: pgmap v779: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:28 smithi189 ceph-mon[32912]: osdmap e514: 8 total, 8 up, 8 in 2024-09-13T13:40:28.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:28 smithi043 ceph-mon[30106]: osdmap e514: 8 total, 8 up, 8 in 2024-09-13T13:40:28.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:28 smithi043 ceph-mon[40255]: osdmap e514: 8 total, 8 up, 8 in 2024-09-13T13:40:29.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: osdmap e515: 8 total, 8 up, 8 in 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[30106]: pgmap v782: 164 pgs: 164 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: osdmap e515: 8 total, 8 up, 8 in 2024-09-13T13:40:29.345 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:29.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:29.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:29 smithi043 ceph-mon[40255]: pgmap v782: 164 pgs: 164 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: osdmap e515: 8 total, 8 up, 8 in 2024-09-13T13:40:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:29.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:29 smithi189 ceph-mon[32912]: pgmap v782: 164 pgs: 164 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:40:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:30 smithi189 ceph-mon[32912]: osdmap e516: 8 total, 8 up, 8 in 2024-09-13T13:40:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[30106]: osdmap e516: 8 total, 8 up, 8 in 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[40255]: osdmap e516: 8 total, 8 up, 8 in 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:40:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:31 smithi189 ceph-mon[32912]: osdmap e517: 8 total, 8 up, 8 in 2024-09-13T13:40:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:31 smithi189 ceph-mon[32912]: pgmap v785: 164 pgs: 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:40:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:31 smithi189 ceph-mon[32912]: osdmap e518: 8 total, 8 up, 8 in 2024-09-13T13:40:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[30106]: osdmap e517: 8 total, 8 up, 8 in 2024-09-13T13:40:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[30106]: pgmap v785: 164 pgs: 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:40:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[30106]: osdmap e518: 8 total, 8 up, 8 in 2024-09-13T13:40:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[40255]: osdmap e517: 8 total, 8 up, 8 in 2024-09-13T13:40:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[40255]: pgmap v785: 164 pgs: 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-76627-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:40:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:31 smithi043 ceph-mon[40255]: osdmap e518: 8 total, 8 up, 8 in 2024-09-13T13:40:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:32 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:32 smithi189 ceph-mon[32912]: osdmap e519: 8 total, 8 up, 8 in 2024-09-13T13:40:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:32 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:32 smithi043 ceph-mon[30106]: osdmap e519: 8 total, 8 up, 8 in 2024-09-13T13:40:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:32 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:32 smithi043 ceph-mon[40255]: osdmap e519: 8 total, 8 up, 8 in 2024-09-13T13:40:33.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[30106]: pgmap v788: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:33.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[30106]: osdmap e520: 8 total, 8 up, 8 in 2024-09-13T13:40:33.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:33.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[40255]: pgmap v788: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[40255]: osdmap e520: 8 total, 8 up, 8 in 2024-09-13T13:40:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:33 smithi189 ceph-mon[32912]: pgmap v788: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:33 smithi189 ceph-mon[32912]: osdmap e520: 8 total, 8 up, 8 in 2024-09-13T13:40:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:35 smithi189 ceph-mon[32912]: osdmap e521: 8 total, 8 up, 8 in 2024-09-13T13:40:35.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:35 smithi189 ceph-mon[32912]: pgmap v791: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:35 smithi043 ceph-mon[30106]: osdmap e521: 8 total, 8 up, 8 in 2024-09-13T13:40:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:35 smithi043 ceph-mon[30106]: pgmap v791: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:35 smithi043 ceph-mon[40255]: osdmap e521: 8 total, 8 up, 8 in 2024-09-13T13:40:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:35 smithi043 ceph-mon[40255]: pgmap v791: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:40:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:40:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:40:36.062 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:40:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:40:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:36 smithi189 ceph-mon[32912]: osdmap e522: 8 total, 8 up, 8 in 2024-09-13T13:40:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:36 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:36.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:36.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[40255]: osdmap e522: 8 total, 8 up, 8 in 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[30106]: osdmap e522: 8 total, 8 up, 8 in 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:37 smithi189 ceph-mon[32912]: osdmap e523: 8 total, 8 up, 8 in 2024-09-13T13:40:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:37 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:37 smithi189 ceph-mon[32912]: pgmap v794: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:37.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:37 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[40255]: osdmap e523: 8 total, 8 up, 8 in 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[40255]: pgmap v794: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[30106]: osdmap e523: 8 total, 8 up, 8 in 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[30106]: pgmap v794: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:37.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:37 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:38.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:38 smithi189 ceph-mon[32912]: osdmap e524: 8 total, 8 up, 8 in 2024-09-13T13:40:38.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:38 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:38.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:38 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:38.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:38.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[30106]: osdmap e524: 8 total, 8 up, 8 in 2024-09-13T13:40:38.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:38.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:38.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:38.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[40255]: osdmap e524: 8 total, 8 up, 8 in 2024-09-13T13:40:38.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:38.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:38 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:39 smithi189 ceph-mon[32912]: osdmap e525: 8 total, 8 up, 8 in 2024-09-13T13:40:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:39 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:39 smithi189 ceph-mon[32912]: pgmap v797: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:39.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[30106]: osdmap e525: 8 total, 8 up, 8 in 2024-09-13T13:40:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[30106]: pgmap v797: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:39.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-107", "overlaypool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:39.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[40255]: osdmap e525: 8 total, 8 up, 8 in 2024-09-13T13:40:39.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:39 smithi043 ceph-mon[40255]: pgmap v797: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:40:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:40 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"}]': finished 2024-09-13T13:40:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:40 smithi189 ceph-mon[32912]: osdmap e526: 8 total, 8 up, 8 in 2024-09-13T13:40:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"} : dispatch 2024-09-13T13:40:40.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"} : dispatch 2024-09-13T13:40:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"}]': finished 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[30106]: osdmap e526: 8 total, 8 up, 8 in 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"} : dispatch 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"} : dispatch 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-107-cache", "mode": "writeback"}]': finished 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[40255]: osdmap e526: 8 total, 8 up, 8 in 2024-09-13T13:40:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"} : dispatch 2024-09-13T13:40:40.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"} : dispatch 2024-09-13T13:40:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"}]': finished 2024-09-13T13:40:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:41 smithi189 ceph-mon[32912]: osdmap e527: 8 total, 8 up, 8 in 2024-09-13T13:40:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:41 smithi189 ceph-mon[32912]: pgmap v800: 236 pgs: 236 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:40:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"}]': finished 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[40255]: osdmap e527: 8 total, 8 up, 8 in 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[40255]: pgmap v800: 236 pgs: 236 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-107"}]': finished 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[30106]: osdmap e527: 8 total, 8 up, 8 in 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3638501' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"} : dispatch 2024-09-13T13:40:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:41 smithi043 ceph-mon[30106]: pgmap v800: 236 pgs: 236 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:40:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:42 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:42.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:42 smithi189 ceph-mon[32912]: osdmap e528: 8 total, 8 up, 8 in 2024-09-13T13:40:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:42 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:42 smithi043 ceph-mon[30106]: osdmap e528: 8 total, 8 up, 8 in 2024-09-13T13:40:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:42 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-107", "tierpool": "test-rados-api-smithi043-76627-107-cache"}]': finished 2024-09-13T13:40:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:42 smithi043 ceph-mon[40255]: osdmap e528: 8 total, 8 up, 8 in 2024-09-13T13:40:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:43 smithi189 ceph-mon[32912]: osdmap e529: 8 total, 8 up, 8 in 2024-09-13T13:40:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:43 smithi189 ceph-mon[32912]: pgmap v803: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:40:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:43 smithi043 ceph-mon[30106]: osdmap e529: 8 total, 8 up, 8 in 2024-09-13T13:40:43.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:43 smithi043 ceph-mon[30106]: pgmap v803: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:40:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:43 smithi043 ceph-mon[40255]: osdmap e529: 8 total, 8 up, 8 in 2024-09-13T13:40:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:43 smithi043 ceph-mon[40255]: pgmap v803: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:40:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:44 smithi189 ceph-mon[32912]: osdmap e530: 8 total, 8 up, 8 in 2024-09-13T13:40:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:44.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[30106]: osdmap e530: 8 total, 8 up, 8 in 2024-09-13T13:40:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[40255]: osdmap e530: 8 total, 8 up, 8 in 2024-09-13T13:40:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:40:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:40:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:45 smithi189 ceph-mon[32912]: osdmap e531: 8 total, 8 up, 8 in 2024-09-13T13:40:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:45.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:45 smithi189 ceph-mon[32912]: pgmap v806: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[40255]: osdmap e531: 8 total, 8 up, 8 in 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[40255]: pgmap v806: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[30106]: osdmap e531: 8 total, 8 up, 8 in 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-mon[30106]: pgmap v806: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:40:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:40:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:40:46.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:40:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:40:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:46 smithi189 ceph-mon[32912]: osdmap e532: 8 total, 8 up, 8 in 2024-09-13T13:40:46.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:46 smithi043 ceph-mon[30106]: osdmap e532: 8 total, 8 up, 8 in 2024-09-13T13:40:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:46 smithi043 ceph-mon[40255]: osdmap e532: 8 total, 8 up, 8 in 2024-09-13T13:40:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:40:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:47 smithi189 ceph-mon[32912]: osdmap e533: 8 total, 8 up, 8 in 2024-09-13T13:40:47.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:47 smithi189 ceph-mon[32912]: pgmap v809: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:47.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:40:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:47 smithi043 ceph-mon[40255]: osdmap e533: 8 total, 8 up, 8 in 2024-09-13T13:40:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:47 smithi043 ceph-mon[40255]: pgmap v809: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:40:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:47 smithi043 ceph-mon[30106]: osdmap e533: 8 total, 8 up, 8 in 2024-09-13T13:40:47.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:47 smithi043 ceph-mon[30106]: pgmap v809: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:48 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:48 smithi189 ceph-mon[32912]: osdmap e534: 8 total, 8 up, 8 in 2024-09-13T13:40:48.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:48 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:48 smithi043 ceph-mon[40255]: osdmap e534: 8 total, 8 up, 8 in 2024-09-13T13:40:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:48 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:40:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:48 smithi043 ceph-mon[30106]: osdmap e534: 8 total, 8 up, 8 in 2024-09-13T13:40:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[30106]: osdmap e535: 8 total, 8 up, 8 in 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[30106]: pgmap v812: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[30106]: osdmap e536: 8 total, 8 up, 8 in 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-109", "overlaypool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[40255]: osdmap e535: 8 total, 8 up, 8 in 2024-09-13T13:40:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[40255]: pgmap v812: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[40255]: osdmap e536: 8 total, 8 up, 8 in 2024-09-13T13:40:49.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-109", "overlaypool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:40:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:49 smithi189 ceph-mon[32912]: osdmap e535: 8 total, 8 up, 8 in 2024-09-13T13:40:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:49 smithi189 ceph-mon[32912]: pgmap v812: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:49.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:49 smithi189 ceph-mon[32912]: osdmap e536: 8 total, 8 up, 8 in 2024-09-13T13:40:49.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-109", "overlaypool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-109", "overlaypool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:51 smithi189 ceph-mon[32912]: osdmap e537: 8 total, 8 up, 8 in 2024-09-13T13:40:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-109-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:51.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:51 smithi189 ceph-mon[32912]: pgmap v815: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-109", "overlaypool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[30106]: osdmap e537: 8 total, 8 up, 8 in 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-109-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[30106]: pgmap v815: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-109", "overlaypool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[40255]: osdmap e537: 8 total, 8 up, 8 in 2024-09-13T13:40:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-109-cache", "mode": "writeback"} : dispatch 2024-09-13T13:40:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:40:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:51 smithi043 ceph-mon[40255]: pgmap v815: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-109-cache", "mode": "writeback"}]': finished 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: osdmap e538: 8 total, 8 up, 8 in 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:40:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-109-cache", "mode": "writeback"}]': finished 2024-09-13T13:40:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: osdmap e538: 8 total, 8 up, 8 in 2024-09-13T13:40:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:40:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:40:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:40:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:52 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:40:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:40:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-109-cache", "mode": "writeback"}]': finished 2024-09-13T13:40:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: osdmap e538: 8 total, 8 up, 8 in 2024-09-13T13:40:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:40:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:40:52.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:40:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:40:52.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:52 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[30106]: osdmap e539: 8 total, 8 up, 8 in 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[30106]: pgmap v818: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[30106]: osdmap e540: 8 total, 8 up, 8 in 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:53.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:40:53.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[40255]: osdmap e539: 8 total, 8 up, 8 in 2024-09-13T13:40:53.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:53.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[40255]: pgmap v818: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:53.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:53.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[40255]: osdmap e540: 8 total, 8 up, 8 in 2024-09-13T13:40:53.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:40:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:53 smithi189 ceph-mon[32912]: osdmap e539: 8 total, 8 up, 8 in 2024-09-13T13:40:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:40:53.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:53 smithi189 ceph-mon[32912]: pgmap v818: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:40:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:40:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:53 smithi189 ceph-mon[32912]: osdmap e540: 8 total, 8 up, 8 in 2024-09-13T13:40:53.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:40:54.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:40:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[30106]: osdmap e541: 8 total, 8 up, 8 in 2024-09-13T13:40:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-13T13:40:54.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:54.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:40:54.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[40255]: osdmap e541: 8 total, 8 up, 8 in 2024-09-13T13:40:54.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-13T13:40:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:54 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:40:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:40:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:54 smithi189 ceph-mon[32912]: osdmap e541: 8 total, 8 up, 8 in 2024-09-13T13:40:54.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-13T13:40:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:55 smithi043 ceph-mon[30106]: pgmap v821: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-13T13:40:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:55 smithi043 ceph-mon[30106]: osdmap e542: 8 total, 8 up, 8 in 2024-09-13T13:40:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:55 smithi043 ceph-mon[40255]: pgmap v821: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-13T13:40:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:55 smithi043 ceph-mon[40255]: osdmap e542: 8 total, 8 up, 8 in 2024-09-13T13:40:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:55 smithi189 ceph-mon[32912]: pgmap v821: 212 pgs: 212 active+clean; 583 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:40:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-13T13:40:55.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:55 smithi189 ceph-mon[32912]: osdmap e542: 8 total, 8 up, 8 in 2024-09-13T13:40:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:40:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:40:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:40:56.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:40:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:40:56.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:56.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:56.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:40:57.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:40:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[30106]: osdmap e543: 8 total, 8 up, 8 in 2024-09-13T13:40:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[30106]: pgmap v824: 212 pgs: 212 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:40:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:40:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[40255]: osdmap e543: 8 total, 8 up, 8 in 2024-09-13T13:40:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:57 smithi043 ceph-mon[40255]: pgmap v824: 212 pgs: 212 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:40:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:40:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:57 smithi189 ceph-mon[32912]: osdmap e543: 8 total, 8 up, 8 in 2024-09-13T13:40:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"} : dispatch 2024-09-13T13:40:57.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:57 smithi189 ceph-mon[32912]: pgmap v824: 212 pgs: 212 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:40:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:58 smithi043 ceph-mon[30106]: osdmap e544: 8 total, 8 up, 8 in 2024-09-13T13:40:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:58.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:58 smithi043 ceph-mon[40255]: osdmap e544: 8 total, 8 up, 8 in 2024-09-13T13:40:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-109", "tierpool": "test-rados-api-smithi043-76627-109-cache"}]': finished 2024-09-13T13:40:58.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:58 smithi189 ceph-mon[32912]: osdmap e544: 8 total, 8 up, 8 in 2024-09-13T13:40:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[30106]: osdmap e545: 8 total, 8 up, 8 in 2024-09-13T13:40:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[30106]: pgmap v827: 180 pgs: 180 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:40:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[30106]: osdmap e546: 8 total, 8 up, 8 in 2024-09-13T13:40:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[40255]: osdmap e545: 8 total, 8 up, 8 in 2024-09-13T13:40:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[40255]: pgmap v827: 180 pgs: 180 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:40:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[40255]: osdmap e546: 8 total, 8 up, 8 in 2024-09-13T13:40:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:40:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:40:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:59 smithi189 ceph-mon[32912]: osdmap e545: 8 total, 8 up, 8 in 2024-09-13T13:40:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:59 smithi189 ceph-mon[32912]: pgmap v827: 180 pgs: 180 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:40:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:59 smithi189 ceph-mon[32912]: osdmap e546: 8 total, 8 up, 8 in 2024-09-13T13:40:59.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:40:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[30106]: osdmap e547: 8 total, 8 up, 8 in 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[30106]: pgmap v830: 172 pgs: 172 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[40255]: osdmap e547: 8 total, 8 up, 8 in 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:41:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:01 smithi043 ceph-mon[40255]: pgmap v830: 172 pgs: 172 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-13T13:41:01.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:41:01.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:01 smithi189 ceph-mon[32912]: osdmap e547: 8 total, 8 up, 8 in 2024-09-13T13:41:01.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"} : dispatch 2024-09-13T13:41:01.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:01 smithi189 ceph-mon[32912]: pgmap v830: 172 pgs: 172 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-13T13:41:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:02 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:41:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:02 smithi043 ceph-mon[30106]: osdmap e548: 8 total, 8 up, 8 in 2024-09-13T13:41:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:02 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:41:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:02 smithi043 ceph-mon[40255]: osdmap e548: 8 total, 8 up, 8 in 2024-09-13T13:41:02.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:02 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:02.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2412951321' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-109"}]': finished 2024-09-13T13:41:02.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:02 smithi189 ceph-mon[32912]: osdmap e548: 8 total, 8 up, 8 in 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[40255]: osdmap e549: 8 total, 8 up, 8 in 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[40255]: pgmap v833: 164 pgs: 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[40255]: osdmap e550: 8 total, 8 up, 8 in 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[30106]: osdmap e549: 8 total, 8 up, 8 in 2024-09-13T13:41:03.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:03.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:03.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[30106]: pgmap v833: 164 pgs: 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-13T13:41:03.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:41:03.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:03 smithi043 ceph-mon[30106]: osdmap e550: 8 total, 8 up, 8 in 2024-09-13T13:41:03.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:03 smithi189 ceph-mon[32912]: osdmap e549: 8 total, 8 up, 8 in 2024-09-13T13:41:03.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:03.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:03.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:03 smithi189 ceph-mon[32912]: pgmap v833: 164 pgs: 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-13T13:41:03.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:41:03.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:03 smithi189 ceph-mon[32912]: osdmap e550: 8 total, 8 up, 8 in 2024-09-13T13:41:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:41:04.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[30106]: osdmap e551: 8 total, 8 up, 8 in 2024-09-13T13:41:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:41:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:04 smithi043 ceph-mon[40255]: osdmap e551: 8 total, 8 up, 8 in 2024-09-13T13:41:04.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/678301397' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:04.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"} : dispatch 2024-09-13T13:41:04.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:04 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-76627-104"}]': finished 2024-09-13T13:41:04.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:04 smithi189 ceph-mon[32912]: osdmap e551: 8 total, 8 up, 8 in 2024-09-13T13:41:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[30106]: pgmap v836: 164 pgs: 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:41:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:41:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-mon[40255]: pgmap v836: 164 pgs: 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:41:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-13T13:41:05.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-mon[32912]: pgmap v836: 164 pgs: 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:41:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:41:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:41:06.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:41:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:41:06.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[30106]: osdmap e552: 8 total, 8 up, 8 in 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[40255]: osdmap e552: 8 total, 8 up, 8 in 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:06.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:06.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-76627-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-13T13:41:06.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:06 smithi189 ceph-mon[32912]: osdmap e552: 8 total, 8 up, 8 in 2024-09-13T13:41:06.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:06.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:06.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:06 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:06.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[40255]: osdmap e553: 8 total, 8 up, 8 in 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[40255]: pgmap v839: 164 pgs: 164 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[40255]: osdmap e554: 8 total, 8 up, 8 in 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[30106]: osdmap e553: 8 total, 8 up, 8 in 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[30106]: pgmap v839: 164 pgs: 164 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:07.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:07 smithi043 ceph-mon[30106]: osdmap e554: 8 total, 8 up, 8 in 2024-09-13T13:41:07.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:07 smithi189 ceph-mon[32912]: osdmap e553: 8 total, 8 up, 8 in 2024-09-13T13:41:07.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:07 smithi189 ceph-mon[32912]: pgmap v839: 164 pgs: 164 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:07.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:07 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:07.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-76627-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:07.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:07 smithi189 ceph-mon[32912]: osdmap e554: 8 total, 8 up, 8 in 2024-09-13T13:41:09.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[30106]: osdmap e555: 8 total, 8 up, 8 in 2024-09-13T13:41:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[30106]: pgmap v842: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[40255]: osdmap e555: 8 total, 8 up, 8 in 2024-09-13T13:41:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:09 smithi043 ceph-mon[40255]: pgmap v842: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:09.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:09 smithi189 ceph-mon[32912]: osdmap e555: 8 total, 8 up, 8 in 2024-09-13T13:41:09.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:09.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:09.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:09 smithi189 ceph-mon[32912]: pgmap v842: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[30106]: osdmap e556: 8 total, 8 up, 8 in 2024-09-13T13:41:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[40255]: osdmap e556: 8 total, 8 up, 8 in 2024-09-13T13:41:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:10 smithi189 ceph-mon[32912]: osdmap e556: 8 total, 8 up, 8 in 2024-09-13T13:41:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:10.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[30106]: osdmap e557: 8 total, 8 up, 8 in 2024-09-13T13:41:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[30106]: pgmap v845: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[40255]: osdmap e557: 8 total, 8 up, 8 in 2024-09-13T13:41:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:11.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:11 smithi043 ceph-mon[40255]: pgmap v845: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:11.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:11.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:11 smithi189 ceph-mon[32912]: osdmap e557: 8 total, 8 up, 8 in 2024-09-13T13:41:11.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:11.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:11.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:11 smithi189 ceph-mon[32912]: pgmap v845: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:12.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"}]': finished 2024-09-13T13:41:12.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:12 smithi189 ceph-mon[32912]: osdmap e558: 8 total, 8 up, 8 in 2024-09-13T13:41:12.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:12.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:12.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:12 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"}]': finished 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[30106]: osdmap e558: 8 total, 8 up, 8 in 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-112"}]': finished 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[40255]: osdmap e558: 8 total, 8 up, 8 in 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:12 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:13.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:13.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:13 smithi189 ceph-mon[32912]: osdmap e559: 8 total, 8 up, 8 in 2024-09-13T13:41:13.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:13.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:13.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:13 smithi189 ceph-mon[32912]: pgmap v848: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[30106]: osdmap e559: 8 total, 8 up, 8 in 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[30106]: pgmap v848: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[40255]: osdmap e559: 8 total, 8 up, 8 in 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"} : dispatch 2024-09-13T13:41:13.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:13 smithi043 ceph-mon[40255]: pgmap v848: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:14.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"}]': finished 2024-09-13T13:41:14.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:14 smithi189 ceph-mon[32912]: osdmap e560: 8 total, 8 up, 8 in 2024-09-13T13:41:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"}]': finished 2024-09-13T13:41:14.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:14 smithi043 ceph-mon[30106]: osdmap e560: 8 total, 8 up, 8 in 2024-09-13T13:41:14.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-112"}]': finished 2024-09-13T13:41:14.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:14 smithi043 ceph-mon[40255]: osdmap e560: 8 total, 8 up, 8 in 2024-09-13T13:41:15.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:15 smithi189 ceph-mon[32912]: osdmap e561: 8 total, 8 up, 8 in 2024-09-13T13:41:15.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:15 smithi189 ceph-mon[32912]: pgmap v851: 172 pgs: 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:15.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:15 smithi043 ceph-mon[30106]: osdmap e561: 8 total, 8 up, 8 in 2024-09-13T13:41:15.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:15 smithi043 ceph-mon[30106]: pgmap v851: 172 pgs: 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:15.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:15 smithi043 ceph-mon[40255]: osdmap e561: 8 total, 8 up, 8 in 2024-09-13T13:41:15.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:15 smithi043 ceph-mon[40255]: pgmap v851: 172 pgs: 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:41:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:41:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:41:16.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:41:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:41:16.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:16 smithi189 ceph-mon[32912]: osdmap e562: 8 total, 8 up, 8 in 2024-09-13T13:41:16.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:16.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:16.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[30106]: osdmap e562: 8 total, 8 up, 8 in 2024-09-13T13:41:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:16.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[40255]: osdmap e562: 8 total, 8 up, 8 in 2024-09-13T13:41:16.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:16.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:16.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:17.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:17.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:17 smithi189 ceph-mon[32912]: osdmap e563: 8 total, 8 up, 8 in 2024-09-13T13:41:17.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:17.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:17.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:17 smithi189 ceph-mon[32912]: pgmap v854: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-13T13:41:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[30106]: osdmap e563: 8 total, 8 up, 8 in 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[30106]: pgmap v854: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[40255]: osdmap e563: 8 total, 8 up, 8 in 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:17.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:17 smithi043 ceph-mon[40255]: pgmap v854: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-13T13:41:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:18 smithi189 ceph-mon[32912]: osdmap e564: 8 total, 8 up, 8 in 2024-09-13T13:41:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:18.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[30106]: osdmap e564: 8 total, 8 up, 8 in 2024-09-13T13:41:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[40255]: osdmap e564: 8 total, 8 up, 8 in 2024-09-13T13:41:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"}]': finished 2024-09-13T13:41:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:19 smithi189 ceph-mon[32912]: osdmap e565: 8 total, 8 up, 8 in 2024-09-13T13:41:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"} : dispatch 2024-09-13T13:41:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"} : dispatch 2024-09-13T13:41:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:19 smithi189 ceph-mon[32912]: pgmap v857: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-13T13:41:19.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:19 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:19.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"}]': finished 2024-09-13T13:41:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"}]': finished 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[30106]: osdmap e565: 8 total, 8 up, 8 in 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"} : dispatch 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"} : dispatch 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[30106]: pgmap v857: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"}]': finished 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-114"}]': finished 2024-09-13T13:41:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[40255]: osdmap e565: 8 total, 8 up, 8 in 2024-09-13T13:41:19.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"} : dispatch 2024-09-13T13:41:19.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"} : dispatch 2024-09-13T13:41:19.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[40255]: pgmap v857: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-13T13:41:19.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:19.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-114", "mode": "writeback"}]': finished 2024-09-13T13:41:20.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:20 smithi189 ceph-mon[32912]: osdmap e566: 8 total, 8 up, 8 in 2024-09-13T13:41:20.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:20.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:20 smithi043 ceph-mon[30106]: osdmap e566: 8 total, 8 up, 8 in 2024-09-13T13:41:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:20 smithi043 ceph-mon[40255]: osdmap e566: 8 total, 8 up, 8 in 2024-09-13T13:41:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:21.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:21.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:21 smithi189 ceph-mon[32912]: osdmap e567: 8 total, 8 up, 8 in 2024-09-13T13:41:21.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:21.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:21.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:21 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:21.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:21 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:21.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:21 smithi189 ceph-mon[32912]: pgmap v860: 204 pgs: 204 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:41:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[30106]: osdmap e567: 8 total, 8 up, 8 in 2024-09-13T13:41:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[30106]: pgmap v860: 204 pgs: 204 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[40255]: osdmap e567: 8 total, 8 up, 8 in 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"} : dispatch 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:21 smithi043 ceph-mon[40255]: pgmap v860: 204 pgs: 204 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-13T13:41:22.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:22 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:22.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"}]': finished 2024-09-13T13:41:22.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:22 smithi189 ceph-mon[32912]: osdmap e568: 8 total, 8 up, 8 in 2024-09-13T13:41:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:22 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"}]': finished 2024-09-13T13:41:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:22 smithi043 ceph-mon[30106]: osdmap e568: 8 total, 8 up, 8 in 2024-09-13T13:41:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:22 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-114"}]': finished 2024-09-13T13:41:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:22 smithi043 ceph-mon[40255]: osdmap e568: 8 total, 8 up, 8 in 2024-09-13T13:41:23.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:23 smithi189 ceph-mon[32912]: osdmap e569: 8 total, 8 up, 8 in 2024-09-13T13:41:23.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:23 smithi189 ceph-mon[32912]: pgmap v863: 172 pgs: 172 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:41:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:23 smithi043 ceph-mon[40255]: osdmap e569: 8 total, 8 up, 8 in 2024-09-13T13:41:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:23 smithi043 ceph-mon[40255]: pgmap v863: 172 pgs: 172 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:41:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:23 smithi043 ceph-mon[30106]: osdmap e569: 8 total, 8 up, 8 in 2024-09-13T13:41:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:23 smithi043 ceph-mon[30106]: pgmap v863: 172 pgs: 172 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:41:24.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:24 smithi189 ceph-mon[32912]: osdmap e570: 8 total, 8 up, 8 in 2024-09-13T13:41:24.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:24.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:24.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:24 smithi043 ceph-mon[40255]: osdmap e570: 8 total, 8 up, 8 in 2024-09-13T13:41:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:24 smithi043 ceph-mon[30106]: osdmap e570: 8 total, 8 up, 8 in 2024-09-13T13:41:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:25.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:25.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:25 smithi189 ceph-mon[32912]: osdmap e571: 8 total, 8 up, 8 in 2024-09-13T13:41:25.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:25 smithi189 ceph-mon[32912]: pgmap v866: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:25.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:25.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:25 smithi043 ceph-mon[30106]: osdmap e571: 8 total, 8 up, 8 in 2024-09-13T13:41:25.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:25 smithi043 ceph-mon[30106]: pgmap v866: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:25.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:25.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:25 smithi043 ceph-mon[40255]: osdmap e571: 8 total, 8 up, 8 in 2024-09-13T13:41:25.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:25 smithi043 ceph-mon[40255]: pgmap v866: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:41:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:41:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:41:26.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:41:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:41:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:26 smithi189 ceph-mon[32912]: osdmap e572: 8 total, 8 up, 8 in 2024-09-13T13:41:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:26.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[30106]: osdmap e572: 8 total, 8 up, 8 in 2024-09-13T13:41:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[40255]: osdmap e572: 8 total, 8 up, 8 in 2024-09-13T13:41:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:27 smithi189 ceph-mon[32912]: osdmap e573: 8 total, 8 up, 8 in 2024-09-13T13:41:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:27.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:27 smithi189 ceph-mon[32912]: pgmap v869: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:41:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[30106]: osdmap e573: 8 total, 8 up, 8 in 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[30106]: pgmap v869: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[40255]: osdmap e573: 8 total, 8 up, 8 in 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:27.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:27 smithi043 ceph-mon[40255]: pgmap v869: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:41:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"}]': finished 2024-09-13T13:41:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:28 smithi189 ceph-mon[32912]: osdmap e574: 8 total, 8 up, 8 in 2024-09-13T13:41:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"} : dispatch 2024-09-13T13:41:28.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"} : dispatch 2024-09-13T13:41:28.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"}]': finished 2024-09-13T13:41:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[40255]: osdmap e574: 8 total, 8 up, 8 in 2024-09-13T13:41:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"} : dispatch 2024-09-13T13:41:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"} : dispatch 2024-09-13T13:41:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-116"}]': finished 2024-09-13T13:41:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[30106]: osdmap e574: 8 total, 8 up, 8 in 2024-09-13T13:41:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"} : dispatch 2024-09-13T13:41:28.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"} : dispatch 2024-09-13T13:41:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:29 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"}]': finished 2024-09-13T13:41:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:29 smithi189 ceph-mon[32912]: osdmap e575: 8 total, 8 up, 8 in 2024-09-13T13:41:29.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:29 smithi189 ceph-mon[32912]: pgmap v872: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:41:29.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.3"} : dispatch 2024-09-13T13:41:29.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:29 smithi189 ceph-mon[32912]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-09-13T13:41:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"}]': finished 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[40255]: osdmap e575: 8 total, 8 up, 8 in 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[40255]: pgmap v872: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.3"} : dispatch 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[40255]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-116", "mode": "writeback"}]': finished 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[30106]: osdmap e575: 8 total, 8 up, 8 in 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[30106]: pgmap v872: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:41:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.3"} : dispatch 2024-09-13T13:41:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:29 smithi043 ceph-mon[30106]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-09-13T13:41:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:30 smithi189 ceph-mon[32912]: 325.3 scrub starts 2024-09-13T13:41:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:30 smithi189 ceph-mon[32912]: 325.3 scrub ok 2024-09-13T13:41:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:30 smithi043 ceph-mon[30106]: 325.3 scrub starts 2024-09-13T13:41:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:30 smithi043 ceph-mon[30106]: 325.3 scrub ok 2024-09-13T13:41:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:30 smithi043 ceph-mon[40255]: 325.3 scrub starts 2024-09-13T13:41:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:30 smithi043 ceph-mon[40255]: 325.3 scrub ok 2024-09-13T13:41:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:31 smithi189 ceph-mon[32912]: pgmap v873: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 610 B/s wr, 2 op/s 2024-09-13T13:41:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:31 smithi043 ceph-mon[30106]: pgmap v873: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 610 B/s wr, 2 op/s 2024-09-13T13:41:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:31 smithi043 ceph-mon[40255]: pgmap v873: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 610 B/s wr, 2 op/s 2024-09-13T13:41:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:33 smithi043 ceph-mon[40255]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 509 B/s wr, 2 op/s 2024-09-13T13:41:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:33 smithi043 ceph-mon[30106]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 509 B/s wr, 2 op/s 2024-09-13T13:41:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:33 smithi189 ceph-mon[32912]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 509 B/s wr, 2 op/s 2024-09-13T13:41:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:35 smithi043 ceph-mon[30106]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 383 B/s wr, 1 op/s 2024-09-13T13:41:35.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:35.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:35 smithi043 ceph-mon[40255]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 383 B/s wr, 1 op/s 2024-09-13T13:41:35.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:35.808 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:35 smithi189 ceph-mon[32912]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 383 B/s wr, 1 op/s 2024-09-13T13:41:35.808 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:41:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:41:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:41:36.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:41:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:41:36.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:37 smithi043 ceph-mon[30106]: pgmap v876: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 340 B/s wr, 1 op/s 2024-09-13T13:41:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:37 smithi043 ceph-mon[40255]: pgmap v876: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 340 B/s wr, 1 op/s 2024-09-13T13:41:37.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:37 smithi189 ceph-mon[32912]: pgmap v876: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 340 B/s wr, 1 op/s 2024-09-13T13:41:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:39 smithi043 ceph-mon[30106]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 306 B/s wr, 1 op/s 2024-09-13T13:41:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:39 smithi043 ceph-mon[40255]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 306 B/s wr, 1 op/s 2024-09-13T13:41:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:39 smithi189 ceph-mon[32912]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 306 B/s wr, 1 op/s 2024-09-13T13:41:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:41 smithi043 ceph-mon[30106]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:41:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:41 smithi043 ceph-mon[40255]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:41:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:41 smithi189 ceph-mon[32912]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:41:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:43 smithi043 ceph-mon[40255]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:41:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:43 smithi043 ceph-mon[30106]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:41:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:43 smithi189 ceph-mon[32912]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:41:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:44 smithi043 ceph-mon[30106]: osdmap e576: 8 total, 8 up, 8 in 2024-09-13T13:41:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:44 smithi043 ceph-mon[40255]: osdmap e576: 8 total, 8 up, 8 in 2024-09-13T13:41:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:44 smithi189 ceph-mon[32912]: osdmap e576: 8 total, 8 up, 8 in 2024-09-13T13:41:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:45.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:41:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[30106]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[30106]: osdmap e577: 8 total, 8 up, 8 in 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[40255]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:45.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[40255]: osdmap e577: 8 total, 8 up, 8 in 2024-09-13T13:41:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:45.815 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:45 smithi189 ceph-mon[32912]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:41:45.815 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:45.815 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:45 smithi189 ceph-mon[32912]: osdmap e577: 8 total, 8 up, 8 in 2024-09-13T13:41:45.815 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:45.815 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"} : dispatch 2024-09-13T13:41:46.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:41:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:41:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"}]': finished 2024-09-13T13:41:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[30106]: osdmap e578: 8 total, 8 up, 8 in 2024-09-13T13:41:46.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"}]': finished 2024-09-13T13:41:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[40255]: osdmap e578: 8 total, 8 up, 8 in 2024-09-13T13:41:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:46 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-116"}]': finished 2024-09-13T13:41:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:46 smithi189 ceph-mon[32912]: osdmap e578: 8 total, 8 up, 8 in 2024-09-13T13:41:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[30106]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[30106]: osdmap e579: 8 total, 8 up, 8 in 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[30106]: osdmap e580: 8 total, 8 up, 8 in 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[40255]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[40255]: osdmap e579: 8 total, 8 up, 8 in 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[40255]: osdmap e580: 8 total, 8 up, 8 in 2024-09-13T13:41:47.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:47.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:47 smithi189 ceph-mon[32912]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:41:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:47 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:47 smithi189 ceph-mon[32912]: osdmap e579: 8 total, 8 up, 8 in 2024-09-13T13:41:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:47 smithi189 ceph-mon[32912]: osdmap e580: 8 total, 8 up, 8 in 2024-09-13T13:41:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:48 smithi189 ceph-mon[32912]: osdmap e581: 8 total, 8 up, 8 in 2024-09-13T13:41:49.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:48 smithi189 ceph-mon[32912]: pgmap v888: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 783 B/s rd, 1 op/s 2024-09-13T13:41:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:48 smithi043 ceph-mon[30106]: osdmap e581: 8 total, 8 up, 8 in 2024-09-13T13:41:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:48 smithi043 ceph-mon[30106]: pgmap v888: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 783 B/s rd, 1 op/s 2024-09-13T13:41:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:48 smithi043 ceph-mon[40255]: osdmap e581: 8 total, 8 up, 8 in 2024-09-13T13:41:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:48 smithi043 ceph-mon[40255]: pgmap v888: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 783 B/s rd, 1 op/s 2024-09-13T13:41:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:50 smithi189 ceph-mon[32912]: osdmap e582: 8 total, 8 up, 8 in 2024-09-13T13:41:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:50.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:50 smithi043 ceph-mon[30106]: osdmap e582: 8 total, 8 up, 8 in 2024-09-13T13:41:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:50 smithi043 ceph-mon[40255]: osdmap e582: 8 total, 8 up, 8 in 2024-09-13T13:41:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: osdmap e583: 8 total, 8 up, 8 in 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: pgmap v891: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 524 B/s wr, 1 op/s 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"}]': finished 2024-09-13T13:41:51.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:51 smithi189 ceph-mon[32912]: osdmap e584: 8 total, 8 up, 8 in 2024-09-13T13:41:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: osdmap e583: 8 total, 8 up, 8 in 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: pgmap v891: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 524 B/s wr, 1 op/s 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"}]': finished 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[30106]: osdmap e584: 8 total, 8 up, 8 in 2024-09-13T13:41:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: osdmap e583: 8 total, 8 up, 8 in 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: pgmap v891: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 524 B/s wr, 1 op/s 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-118"}]': finished 2024-09-13T13:41:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:51 smithi043 ceph-mon[40255]: osdmap e584: 8 total, 8 up, 8 in 2024-09-13T13:41:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"} : dispatch 2024-09-13T13:41:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"} : dispatch 2024-09-13T13:41:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:52 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:52 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"}]': finished 2024-09-13T13:41:52.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:52 smithi189 ceph-mon[32912]: osdmap e585: 8 total, 8 up, 8 in 2024-09-13T13:41:52.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"} : dispatch 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"} : dispatch 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"}]': finished 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[30106]: osdmap e585: 8 total, 8 up, 8 in 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"} : dispatch 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"} : dispatch 2024-09-13T13:41:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:41:52.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:41:52.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-118", "mode": "writeback"}]': finished 2024-09-13T13:41:52.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:52 smithi043 ceph-mon[40255]: osdmap e585: 8 total, 8 up, 8 in 2024-09-13T13:41:53.319 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:53 smithi189 ceph-mon[32912]: pgmap v894: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:53.319 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:53 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:41:53.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:53 smithi043 ceph-mon[30106]: pgmap v894: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:53.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:53 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:41:53.341 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:53 smithi043 ceph-mon[40255]: pgmap v894: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-13T13:41:53.342 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:53 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:41:54.329 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:54 smithi189 ceph-mon[32912]: osdmap e586: 8 total, 8 up, 8 in 2024-09-13T13:41:54.329 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:54.329 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:54.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:54 smithi043 ceph-mon[30106]: osdmap e586: 8 total, 8 up, 8 in 2024-09-13T13:41:54.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:54.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:54.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:54 smithi043 ceph-mon[40255]: osdmap e586: 8 total, 8 up, 8 in 2024-09-13T13:41:54.462 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:54.462 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:41:55.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:55.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[40255]: osdmap e587: 8 total, 8 up, 8 in 2024-09-13T13:41:55.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:55.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[30106]: osdmap e587: 8 total, 8 up, 8 in 2024-09-13T13:41:55.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:55.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:55.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[30106]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:41:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:55 smithi189 ceph-mon[32912]: osdmap e587: 8 total, 8 up, 8 in 2024-09-13T13:41:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:55.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:55 smithi189 ceph-mon[32912]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"} : dispatch 2024-09-13T13:41:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-mon[40255]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:41:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:41:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:41:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:41:56.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:41:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:41:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:56 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"}]': finished 2024-09-13T13:41:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:56 smithi189 ceph-mon[32912]: osdmap e588: 8 total, 8 up, 8 in 2024-09-13T13:41:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:56.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:56 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:56.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:56 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:56.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:56.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"}]': finished 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[40255]: osdmap e588: 8 total, 8 up, 8 in 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-118"}]': finished 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[30106]: osdmap e588: 8 total, 8 up, 8 in 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:56.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:56 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:57 smithi189 ceph-mon[32912]: osdmap e589: 8 total, 8 up, 8 in 2024-09-13T13:41:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:57 smithi189 ceph-mon[32912]: pgmap v900: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:57 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:57 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[40255]: osdmap e589: 8 total, 8 up, 8 in 2024-09-13T13:41:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[40255]: pgmap v900: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[30106]: osdmap e589: 8 total, 8 up, 8 in 2024-09-13T13:41:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[30106]: pgmap v900: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:57 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:58 smithi189 ceph-mon[32912]: osdmap e590: 8 total, 8 up, 8 in 2024-09-13T13:41:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:58 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:41:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:58 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:41:58.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:58 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[40255]: osdmap e590: 8 total, 8 up, 8 in 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[30106]: osdmap e590: 8 total, 8 up, 8 in 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:41:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:41:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:41:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:58 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:41:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:59 smithi189 ceph-mon[32912]: osdmap e591: 8 total, 8 up, 8 in 2024-09-13T13:41:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:59.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:41:59 smithi189 ceph-mon[32912]: pgmap v903: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[40255]: osdmap e591: 8 total, 8 up, 8 in 2024-09-13T13:41:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[40255]: pgmap v903: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:41:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:41:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[30106]: osdmap e591: 8 total, 8 up, 8 in 2024-09-13T13:41:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:41:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:41:59 smithi043 ceph-mon[30106]: pgmap v903: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:00 smithi189 ceph-mon[32912]: osdmap e592: 8 total, 8 up, 8 in 2024-09-13T13:42:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[30106]: osdmap e592: 8 total, 8 up, 8 in 2024-09-13T13:42:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:00.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:00.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[40255]: osdmap e592: 8 total, 8 up, 8 in 2024-09-13T13:42:00.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:00.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"}]': finished 2024-09-13T13:42:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:01 smithi189 ceph-mon[32912]: osdmap e593: 8 total, 8 up, 8 in 2024-09-13T13:42:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"} : dispatch 2024-09-13T13:42:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"} : dispatch 2024-09-13T13:42:01.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:01 smithi189 ceph-mon[32912]: pgmap v906: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"}]': finished 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[30106]: osdmap e593: 8 total, 8 up, 8 in 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"} : dispatch 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"} : dispatch 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[30106]: pgmap v906: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-120"}]': finished 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[40255]: osdmap e593: 8 total, 8 up, 8 in 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"} : dispatch 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"} : dispatch 2024-09-13T13:42:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:01 smithi043 ceph-mon[40255]: pgmap v906: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"}]': finished 2024-09-13T13:42:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: osdmap e594: 8 total, 8 up, 8 in 2024-09-13T13:42:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: osdmap e595: 8 total, 8 up, 8 in 2024-09-13T13:42:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:02.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"}]': finished 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: osdmap e594: 8 total, 8 up, 8 in 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: osdmap e595: 8 total, 8 up, 8 in 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-120", "mode": "writeback"}]': finished 2024-09-13T13:42:02.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: osdmap e594: 8 total, 8 up, 8 in 2024-09-13T13:42:02.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:02.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:02.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:02.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: osdmap e595: 8 total, 8 up, 8 in 2024-09-13T13:42:02.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:02.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"} : dispatch 2024-09-13T13:42:03.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[30106]: pgmap v909: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:03.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:03.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"}]': finished 2024-09-13T13:42:03.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[30106]: osdmap e596: 8 total, 8 up, 8 in 2024-09-13T13:42:03.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[40255]: pgmap v909: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:03.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:03.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"}]': finished 2024-09-13T13:42:03.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:03 smithi043 ceph-mon[40255]: osdmap e596: 8 total, 8 up, 8 in 2024-09-13T13:42:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:03 smithi189 ceph-mon[32912]: pgmap v909: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:03 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-120"}]': finished 2024-09-13T13:42:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:03 smithi189 ceph-mon[32912]: osdmap e596: 8 total, 8 up, 8 in 2024-09-13T13:42:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:05 smithi189 ceph-mon[32912]: osdmap e597: 8 total, 8 up, 8 in 2024-09-13T13:42:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:05 smithi189 ceph-mon[32912]: pgmap v912: 172 pgs: 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:05 smithi043 ceph-mon[30106]: osdmap e597: 8 total, 8 up, 8 in 2024-09-13T13:42:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:05 smithi043 ceph-mon[30106]: pgmap v912: 172 pgs: 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:05 smithi043 ceph-mon[40255]: osdmap e597: 8 total, 8 up, 8 in 2024-09-13T13:42:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:05 smithi043 ceph-mon[40255]: pgmap v912: 172 pgs: 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:42:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:42:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:42:06.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:42:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:42:06.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:06 smithi189 ceph-mon[32912]: osdmap e598: 8 total, 8 up, 8 in 2024-09-13T13:42:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:06 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:06.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[40255]: osdmap e598: 8 total, 8 up, 8 in 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[30106]: osdmap e598: 8 total, 8 up, 8 in 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:06.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:07 smithi189 ceph-mon[32912]: osdmap e599: 8 total, 8 up, 8 in 2024-09-13T13:42:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:07 smithi189 ceph-mon[32912]: pgmap v915: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[30106]: osdmap e599: 8 total, 8 up, 8 in 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[30106]: pgmap v915: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[40255]: osdmap e599: 8 total, 8 up, 8 in 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:07.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:07 smithi043 ceph-mon[40255]: pgmap v915: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:08 smithi189 ceph-mon[32912]: osdmap e600: 8 total, 8 up, 8 in 2024-09-13T13:42:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:08.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:08.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:08.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[40255]: osdmap e600: 8 total, 8 up, 8 in 2024-09-13T13:42:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:08.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[30106]: osdmap e600: 8 total, 8 up, 8 in 2024-09-13T13:42:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"}]': finished 2024-09-13T13:42:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:09 smithi189 ceph-mon[32912]: osdmap e601: 8 total, 8 up, 8 in 2024-09-13T13:42:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"} : dispatch 2024-09-13T13:42:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"} : dispatch 2024-09-13T13:42:09.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:09 smithi189 ceph-mon[32912]: pgmap v918: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:09.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"}]': finished 2024-09-13T13:42:09.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[40255]: osdmap e601: 8 total, 8 up, 8 in 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"} : dispatch 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"} : dispatch 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[40255]: pgmap v918: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-122"}]': finished 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[30106]: osdmap e601: 8 total, 8 up, 8 in 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"} : dispatch 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"} : dispatch 2024-09-13T13:42:09.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:09 smithi043 ceph-mon[30106]: pgmap v918: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:10 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"}]': finished 2024-09-13T13:42:10.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:10 smithi189 ceph-mon[32912]: osdmap e602: 8 total, 8 up, 8 in 2024-09-13T13:42:10.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:10.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"}]': finished 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[40255]: osdmap e602: 8 total, 8 up, 8 in 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-122", "mode": "writeback"}]': finished 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[30106]: osdmap e602: 8 total, 8 up, 8 in 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:11 smithi189 ceph-mon[32912]: osdmap e603: 8 total, 8 up, 8 in 2024-09-13T13:42:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:11.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:11 smithi189 ceph-mon[32912]: pgmap v921: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-13T13:42:11.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:11 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[40255]: osdmap e603: 8 total, 8 up, 8 in 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[40255]: pgmap v921: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[30106]: osdmap e603: 8 total, 8 up, 8 in 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"} : dispatch 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[30106]: pgmap v921: 204 pgs: 204 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-13T13:42:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:11 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"}]': finished 2024-09-13T13:42:12.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:12 smithi189 ceph-mon[32912]: osdmap e604: 8 total, 8 up, 8 in 2024-09-13T13:42:12.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"}]': finished 2024-09-13T13:42:12.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:12 smithi043 ceph-mon[40255]: osdmap e604: 8 total, 8 up, 8 in 2024-09-13T13:42:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-122"}]': finished 2024-09-13T13:42:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:12 smithi043 ceph-mon[30106]: osdmap e604: 8 total, 8 up, 8 in 2024-09-13T13:42:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:13 smithi189 ceph-mon[32912]: osdmap e605: 8 total, 8 up, 8 in 2024-09-13T13:42:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:13 smithi189 ceph-mon[32912]: pgmap v924: 172 pgs: 172 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:42:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:13 smithi043 ceph-mon[40255]: osdmap e605: 8 total, 8 up, 8 in 2024-09-13T13:42:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:13 smithi043 ceph-mon[40255]: pgmap v924: 172 pgs: 172 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:42:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:13 smithi043 ceph-mon[30106]: osdmap e605: 8 total, 8 up, 8 in 2024-09-13T13:42:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:13 smithi043 ceph-mon[30106]: pgmap v924: 172 pgs: 172 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:42:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:14 smithi189 ceph-mon[32912]: osdmap e606: 8 total, 8 up, 8 in 2024-09-13T13:42:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:14.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:14 smithi043 ceph-mon[30106]: osdmap e606: 8 total, 8 up, 8 in 2024-09-13T13:42:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:14 smithi043 ceph-mon[40255]: osdmap e606: 8 total, 8 up, 8 in 2024-09-13T13:42:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:15 smithi189 ceph-mon[32912]: osdmap e607: 8 total, 8 up, 8 in 2024-09-13T13:42:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:15 smithi189 ceph-mon[32912]: pgmap v927: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:15 smithi043 ceph-mon[30106]: osdmap e607: 8 total, 8 up, 8 in 2024-09-13T13:42:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:15 smithi043 ceph-mon[30106]: pgmap v927: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:15 smithi043 ceph-mon[40255]: osdmap e607: 8 total, 8 up, 8 in 2024-09-13T13:42:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:15 smithi043 ceph-mon[40255]: pgmap v927: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:42:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:42:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:42:16.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:42:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:42:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:16 smithi189 ceph-mon[32912]: osdmap e608: 8 total, 8 up, 8 in 2024-09-13T13:42:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:16.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:16 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:16.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[30106]: osdmap e608: 8 total, 8 up, 8 in 2024-09-13T13:42:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[40255]: osdmap e608: 8 total, 8 up, 8 in 2024-09-13T13:42:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:16 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:17 smithi189 ceph-mon[32912]: osdmap e609: 8 total, 8 up, 8 in 2024-09-13T13:42:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:17 smithi189 ceph-mon[32912]: pgmap v930: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:42:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[30106]: osdmap e609: 8 total, 8 up, 8 in 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[30106]: pgmap v930: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[40255]: osdmap e609: 8 total, 8 up, 8 in 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:17 smithi043 ceph-mon[40255]: pgmap v930: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:42:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"}]': finished 2024-09-13T13:42:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:18 smithi189 ceph-mon[32912]: osdmap e610: 8 total, 8 up, 8 in 2024-09-13T13:42:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"} : dispatch 2024-09-13T13:42:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"} : dispatch 2024-09-13T13:42:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:18 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"}]': finished 2024-09-13T13:42:18.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:18 smithi189 ceph-mon[32912]: osdmap e611: 8 total, 8 up, 8 in 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"}]': finished 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[30106]: osdmap e610: 8 total, 8 up, 8 in 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"} : dispatch 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"} : dispatch 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"}]': finished 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[30106]: osdmap e611: 8 total, 8 up, 8 in 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-124"}]': finished 2024-09-13T13:42:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[40255]: osdmap e610: 8 total, 8 up, 8 in 2024-09-13T13:42:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"} : dispatch 2024-09-13T13:42:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"} : dispatch 2024-09-13T13:42:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-124", "mode": "writeback"}]': finished 2024-09-13T13:42:18.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:18 smithi043 ceph-mon[40255]: osdmap e611: 8 total, 8 up, 8 in 2024-09-13T13:42:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:19 smithi189 ceph-mon[32912]: pgmap v933: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:42:19.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:19 smithi043 ceph-mon[30106]: pgmap v933: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:42:19.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:19 smithi043 ceph-mon[40255]: pgmap v933: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-13T13:42:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:20 smithi189 ceph-mon[32912]: osdmap e612: 8 total, 8 up, 8 in 2024-09-13T13:42:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:20 smithi043 ceph-mon[30106]: osdmap e612: 8 total, 8 up, 8 in 2024-09-13T13:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:20 smithi043 ceph-mon[40255]: osdmap e612: 8 total, 8 up, 8 in 2024-09-13T13:42:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:21.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:21.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:21 smithi189 ceph-mon[32912]: osdmap e613: 8 total, 8 up, 8 in 2024-09-13T13:42:21.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:21.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:21.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:21 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:21.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:21 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:21.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:21 smithi189 ceph-mon[32912]: pgmap v936: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[30106]: osdmap e613: 8 total, 8 up, 8 in 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[30106]: pgmap v936: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-13T13:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:21.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[40255]: osdmap e613: 8 total, 8 up, 8 in 2024-09-13T13:42:21.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:21.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"} : dispatch 2024-09-13T13:42:21.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:21.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:21.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:21 smithi043 ceph-mon[40255]: pgmap v936: 204 pgs: 204 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-13T13:42:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:22 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"}]': finished 2024-09-13T13:42:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:22 smithi189 ceph-mon[32912]: osdmap e614: 8 total, 8 up, 8 in 2024-09-13T13:42:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:22 smithi189 ceph-mon[32912]: osdmap e615: 8 total, 8 up, 8 in 2024-09-13T13:42:22.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"}]': finished 2024-09-13T13:42:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[30106]: osdmap e614: 8 total, 8 up, 8 in 2024-09-13T13:42:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[30106]: osdmap e615: 8 total, 8 up, 8 in 2024-09-13T13:42:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-124"}]': finished 2024-09-13T13:42:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[40255]: osdmap e614: 8 total, 8 up, 8 in 2024-09-13T13:42:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:22 smithi043 ceph-mon[40255]: osdmap e615: 8 total, 8 up, 8 in 2024-09-13T13:42:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:23 smithi189 ceph-mon[32912]: pgmap v939: 172 pgs: 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:23 smithi189 ceph-mon[32912]: osdmap e616: 8 total, 8 up, 8 in 2024-09-13T13:42:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:23.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[30106]: pgmap v939: 172 pgs: 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[30106]: osdmap e616: 8 total, 8 up, 8 in 2024-09-13T13:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[40255]: pgmap v939: 172 pgs: 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[40255]: osdmap e616: 8 total, 8 up, 8 in 2024-09-13T13:42:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:25 smithi189 ceph-mon[32912]: osdmap e617: 8 total, 8 up, 8 in 2024-09-13T13:42:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:25 smithi189 ceph-mon[32912]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[30106]: osdmap e617: 8 total, 8 up, 8 in 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[30106]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[40255]: osdmap e617: 8 total, 8 up, 8 in 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-mon[40255]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:42:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:42:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:42:26.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:42:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:42:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:26.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:26 smithi189 ceph-mon[32912]: osdmap e618: 8 total, 8 up, 8 in 2024-09-13T13:42:26.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:26.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:26.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:26 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[30106]: osdmap e618: 8 total, 8 up, 8 in 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[40255]: osdmap e618: 8 total, 8 up, 8 in 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:26 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"}]': finished 2024-09-13T13:42:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: osdmap e619: 8 total, 8 up, 8 in 2024-09-13T13:42:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"} : dispatch 2024-09-13T13:42:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"} : dispatch 2024-09-13T13:42:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: pgmap v945: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"}]': finished 2024-09-13T13:42:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:27 smithi189 ceph-mon[32912]: osdmap e620: 8 total, 8 up, 8 in 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"}]': finished 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: osdmap e619: 8 total, 8 up, 8 in 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"} : dispatch 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"} : dispatch 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: pgmap v945: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"}]': finished 2024-09-13T13:42:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[30106]: osdmap e620: 8 total, 8 up, 8 in 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-126"}]': finished 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: osdmap e619: 8 total, 8 up, 8 in 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"} : dispatch 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"} : dispatch 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: pgmap v945: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-126", "mode": "writeback"}]': finished 2024-09-13T13:42:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:27 smithi043 ceph-mon[40255]: osdmap e620: 8 total, 8 up, 8 in 2024-09-13T13:42:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:28.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:28.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:28.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:29 smithi189 ceph-mon[32912]: osdmap e621: 8 total, 8 up, 8 in 2024-09-13T13:42:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:29 smithi189 ceph-mon[32912]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[30106]: osdmap e621: 8 total, 8 up, 8 in 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[30106]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[40255]: osdmap e621: 8 total, 8 up, 8 in 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"} : dispatch 2024-09-13T13:42:29.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:29 smithi043 ceph-mon[40255]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:30.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:42:30.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9058 ms) 2024-09-13T13:42:30.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-13T13:42:30.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:42:30.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6042 ms) 2024-09-13T13:42:30.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5039 ms) 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (26082 ms) 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14052 ms) 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12079 ms) 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-13T13:42:30.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000012 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12080 ms) 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6040 ms) 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (555154 ms total) 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1035 ms) 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11036 ms) 2024-09-13T13:42:30.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18076 ms) 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30149 ms total) 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7040 ms) 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8080 ms) 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrub... 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24154 ms) 2024-09-13T13:42:30.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9540 ms) 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7970 ms) 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8052 ms) 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (9950 ms) 2024-09-13T13:42:30.052 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-13T13:42:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:30 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"}]': finished 2024-09-13T13:42:30.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:30 smithi189 ceph-mon[32912]: osdmap e622: 8 total, 8 up, 8 in 2024-09-13T13:42:30.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:30 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:30.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"}]': finished 2024-09-13T13:42:30.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:30 smithi043 ceph-mon[30106]: osdmap e622: 8 total, 8 up, 8 in 2024-09-13T13:42:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:30 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-126"}]': finished 2024-09-13T13:42:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:30 smithi043 ceph-mon[40255]: osdmap e622: 8 total, 8 up, 8 in 2024-09-13T13:42:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:31 smithi189 ceph-mon[32912]: osdmap e623: 8 total, 8 up, 8 in 2024-09-13T13:42:31.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:31 smithi189 ceph-mon[32912]: pgmap v951: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:31 smithi043 ceph-mon[30106]: osdmap e623: 8 total, 8 up, 8 in 2024-09-13T13:42:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:31 smithi043 ceph-mon[30106]: pgmap v951: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:31 smithi043 ceph-mon[40255]: osdmap e623: 8 total, 8 up, 8 in 2024-09-13T13:42:31.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:31 smithi043 ceph-mon[40255]: pgmap v951: 172 pgs: 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:32 smithi189 ceph-mon[32912]: osdmap e624: 8 total, 8 up, 8 in 2024-09-13T13:42:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:32 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:32.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:32 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:32.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:32 smithi043 ceph-mon[30106]: osdmap e624: 8 total, 8 up, 8 in 2024-09-13T13:42:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:32 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:32 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:32 smithi043 ceph-mon[40255]: osdmap e624: 8 total, 8 up, 8 in 2024-09-13T13:42:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:32 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:32.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:32 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:33.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:33.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[30106]: osdmap e625: 8 total, 8 up, 8 in 2024-09-13T13:42:33.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:33.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:33.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[30106]: pgmap v954: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:33.392 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[40255]: osdmap e625: 8 total, 8 up, 8 in 2024-09-13T13:42:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:33.393 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:33 smithi043 ceph-mon[40255]: pgmap v954: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:33 smithi189 ceph-mon[32912]: osdmap e625: 8 total, 8 up, 8 in 2024-09-13T13:42:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:33.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:33.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:33 smithi189 ceph-mon[32912]: pgmap v954: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:34.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:34.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:34 smithi189 ceph-mon[32912]: osdmap e626: 8 total, 8 up, 8 in 2024-09-13T13:42:34.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:34.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:34.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:34.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[30106]: osdmap e626: 8 total, 8 up, 8 in 2024-09-13T13:42:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[40255]: osdmap e626: 8 total, 8 up, 8 in 2024-09-13T13:42:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"}]': finished 2024-09-13T13:42:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-mon[32912]: osdmap e627: 8 total, 8 up, 8 in 2024-09-13T13:42:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"} : dispatch 2024-09-13T13:42:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"} : dispatch 2024-09-13T13:42:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-mon[32912]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:35.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"}]': finished 2024-09-13T13:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"}]': finished 2024-09-13T13:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[30106]: osdmap e627: 8 total, 8 up, 8 in 2024-09-13T13:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"} : dispatch 2024-09-13T13:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"} : dispatch 2024-09-13T13:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[30106]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"}]': finished 2024-09-13T13:42:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-128"}]': finished 2024-09-13T13:42:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[40255]: osdmap e627: 8 total, 8 up, 8 in 2024-09-13T13:42:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"} : dispatch 2024-09-13T13:42:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"} : dispatch 2024-09-13T13:42:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[40255]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-128", "mode": "writeback"}]': finished 2024-09-13T13:42:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:42:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:42:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:42:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:36 smithi189 ceph-mon[32912]: osdmap e628: 8 total, 8 up, 8 in 2024-09-13T13:42:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:36 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:36 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:36.149 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:42:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:42:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:36 smithi043 ceph-mon[30106]: osdmap e628: 8 total, 8 up, 8 in 2024-09-13T13:42:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:36 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:36 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:36 smithi043 ceph-mon[40255]: osdmap e628: 8 total, 8 up, 8 in 2024-09-13T13:42:36.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:36 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:36.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:36 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:37.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:37 smithi189 ceph-mon[32912]: pgmap v959: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 470 B/s wr, 1 op/s 2024-09-13T13:42:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:37 smithi043 ceph-mon[40255]: pgmap v959: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 470 B/s wr, 1 op/s 2024-09-13T13:42:37.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:37 smithi043 ceph-mon[30106]: pgmap v959: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 470 B/s wr, 1 op/s 2024-09-13T13:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:39 smithi043 ceph-mon[40255]: pgmap v960: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-09-13T13:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:39 smithi043 ceph-mon[30106]: pgmap v960: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-09-13T13:42:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:39 smithi189 ceph-mon[32912]: pgmap v960: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-09-13T13:42:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:40.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[30106]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 278 B/s wr, 2 op/s 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[30106]: osdmap e629: 8 total, 8 up, 8 in 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[40255]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 278 B/s wr, 2 op/s 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[40255]: osdmap e629: 8 total, 8 up, 8 in 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:41 smithi189 ceph-mon[32912]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 278 B/s wr, 2 op/s 2024-09-13T13:42:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:41 smithi189 ceph-mon[32912]: osdmap e629: 8 total, 8 up, 8 in 2024-09-13T13:42:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"} : dispatch 2024-09-13T13:42:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:42 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"}]': finished 2024-09-13T13:42:42.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:42 smithi189 ceph-mon[32912]: osdmap e630: 8 total, 8 up, 8 in 2024-09-13T13:42:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:42 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"}]': finished 2024-09-13T13:42:42.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:42 smithi043 ceph-mon[40255]: osdmap e630: 8 total, 8 up, 8 in 2024-09-13T13:42:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:42 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-128"}]': finished 2024-09-13T13:42:42.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:42 smithi043 ceph-mon[30106]: osdmap e630: 8 total, 8 up, 8 in 2024-09-13T13:42:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:43 smithi189 ceph-mon[32912]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 278 B/s wr, 2 op/s 2024-09-13T13:42:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:43 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:43 smithi189 ceph-mon[32912]: osdmap e631: 8 total, 8 up, 8 in 2024-09-13T13:42:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:43 smithi043 ceph-mon[40255]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 278 B/s wr, 2 op/s 2024-09-13T13:42:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:43 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:43 smithi043 ceph-mon[40255]: osdmap e631: 8 total, 8 up, 8 in 2024-09-13T13:42:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:43 smithi043 ceph-mon[30106]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 278 B/s wr, 2 op/s 2024-09-13T13:42:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:43 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:43 smithi043 ceph-mon[30106]: osdmap e631: 8 total, 8 up, 8 in 2024-09-13T13:42:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:44 smithi189 ceph-mon[32912]: osdmap e632: 8 total, 8 up, 8 in 2024-09-13T13:42:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:44.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:44.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:44 smithi043 ceph-mon[40255]: osdmap e632: 8 total, 8 up, 8 in 2024-09-13T13:42:44.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:44.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:44.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:44 smithi043 ceph-mon[30106]: osdmap e632: 8 total, 8 up, 8 in 2024-09-13T13:42:44.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:44.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:45.869 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:45 smithi189 ceph-mon[32912]: pgmap v967: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:45.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:45.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:45 smithi189 ceph-mon[32912]: osdmap e633: 8 total, 8 up, 8 in 2024-09-13T13:42:45.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:45.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:42:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:42:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[40255]: pgmap v967: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[40255]: osdmap e633: 8 total, 8 up, 8 in 2024-09-13T13:42:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[30106]: pgmap v967: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[30106]: osdmap e633: 8 total, 8 up, 8 in 2024-09-13T13:42:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:45.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:46.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:42:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:42:46.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:46.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:46 smithi189 ceph-mon[32912]: osdmap e634: 8 total, 8 up, 8 in 2024-09-13T13:42:46.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:46.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:46.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:46 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:46.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[30106]: osdmap e634: 8 total, 8 up, 8 in 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[40255]: osdmap e634: 8 total, 8 up, 8 in 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:46.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:46 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:47 smithi189 ceph-mon[32912]: pgmap v970: 204 pgs: 204 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:42:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"}]': finished 2024-09-13T13:42:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:47 smithi189 ceph-mon[32912]: osdmap e635: 8 total, 8 up, 8 in 2024-09-13T13:42:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"} : dispatch 2024-09-13T13:42:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"} : dispatch 2024-09-13T13:42:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[30106]: pgmap v970: 204 pgs: 204 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"}]': finished 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[30106]: osdmap e635: 8 total, 8 up, 8 in 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"} : dispatch 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"} : dispatch 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[40255]: pgmap v970: 204 pgs: 204 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-130"}]': finished 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[40255]: osdmap e635: 8 total, 8 up, 8 in 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"} : dispatch 2024-09-13T13:42:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"} : dispatch 2024-09-13T13:42:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:48 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"}]': finished 2024-09-13T13:42:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:48 smithi189 ceph-mon[32912]: osdmap e636: 8 total, 8 up, 8 in 2024-09-13T13:42:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:48 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"}]': finished 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[30106]: osdmap e636: 8 total, 8 up, 8 in 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-130", "mode": "writeback"}]': finished 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[40255]: osdmap e636: 8 total, 8 up, 8 in 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:49 smithi189 ceph-mon[32912]: pgmap v973: 204 pgs: 204 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:49 smithi189 ceph-mon[32912]: osdmap e637: 8 total, 8 up, 8 in 2024-09-13T13:42:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[30106]: pgmap v973: 204 pgs: 204 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[30106]: osdmap e637: 8 total, 8 up, 8 in 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[40255]: pgmap v973: 204 pgs: 204 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[40255]: osdmap e637: 8 total, 8 up, 8 in 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"} : dispatch 2024-09-13T13:42:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:50 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"}]': finished 2024-09-13T13:42:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:50 smithi189 ceph-mon[32912]: osdmap e638: 8 total, 8 up, 8 in 2024-09-13T13:42:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:50 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"}]': finished 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[30106]: osdmap e638: 8 total, 8 up, 8 in 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-130"}]': finished 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[40255]: osdmap e638: 8 total, 8 up, 8 in 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:42:51.799 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:51 smithi189 ceph-mon[32912]: pgmap v976: 204 pgs: 204 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-13T13:42:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:51 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:51 smithi189 ceph-mon[32912]: osdmap e639: 8 total, 8 up, 8 in 2024-09-13T13:42:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:51 smithi043 ceph-mon[30106]: pgmap v976: 204 pgs: 204 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-13T13:42:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:51 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:51 smithi043 ceph-mon[30106]: osdmap e639: 8 total, 8 up, 8 in 2024-09-13T13:42:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:51 smithi043 ceph-mon[40255]: pgmap v976: 204 pgs: 204 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-13T13:42:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:51 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:51 smithi043 ceph-mon[40255]: osdmap e639: 8 total, 8 up, 8 in 2024-09-13T13:42:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:52 smithi189 ceph-mon[32912]: osdmap e640: 8 total, 8 up, 8 in 2024-09-13T13:42:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:52.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:52 smithi043 ceph-mon[30106]: osdmap e640: 8 total, 8 up, 8 in 2024-09-13T13:42:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:52 smithi043 ceph-mon[40255]: osdmap e640: 8 total, 8 up, 8 in 2024-09-13T13:42:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:42:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:53 smithi189 ceph-mon[32912]: pgmap v979: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:53 smithi189 ceph-mon[32912]: osdmap e641: 8 total, 8 up, 8 in 2024-09-13T13:42:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[30106]: pgmap v979: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[30106]: osdmap e641: 8 total, 8 up, 8 in 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[40255]: pgmap v979: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[40255]: osdmap e641: 8 total, 8 up, 8 in 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:42:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:54 smithi189 ceph-mon[32912]: osdmap e642: 8 total, 8 up, 8 in 2024-09-13T13:42:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[40255]: osdmap e642: 8 total, 8 up, 8 in 2024-09-13T13:42:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:42:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[30106]: osdmap e642: 8 total, 8 up, 8 in 2024-09-13T13:42:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:55.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:55.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:42:55.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: osdmap e643: 8 total, 8 up, 8 in 2024-09-13T13:42:55.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"} : dispatch 2024-09-13T13:42:55.872 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"} : dispatch 2024-09-13T13:42:55.872 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:55.872 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"}]': finished 2024-09-13T13:42:55.872 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-mon[32912]: osdmap e644: 8 total, 8 up, 8 in 2024-09-13T13:42:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:42:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: osdmap e643: 8 total, 8 up, 8 in 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"} : dispatch 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"} : dispatch 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"}]': finished 2024-09-13T13:42:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[40255]: osdmap e644: 8 total, 8 up, 8 in 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: osdmap e643: 8 total, 8 up, 8 in 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"} : dispatch 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"} : dispatch 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-132", "mode": "writeback"}]': finished 2024-09-13T13:42:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:55 smithi043 ceph-mon[30106]: osdmap e644: 8 total, 8 up, 8 in 2024-09-13T13:42:56.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:42:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:42:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:42:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:57 smithi189 ceph-mon[32912]: pgmap v985: 204 pgs: 204 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:57 smithi189 ceph-mon[32912]: osdmap e645: 8 total, 8 up, 8 in 2024-09-13T13:42:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:57 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:57 smithi043 ceph-mon[30106]: pgmap v985: 204 pgs: 204 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:57 smithi043 ceph-mon[30106]: osdmap e645: 8 total, 8 up, 8 in 2024-09-13T13:42:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:57 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:57 smithi043 ceph-mon[40255]: pgmap v985: 204 pgs: 204 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:57 smithi043 ceph-mon[40255]: osdmap e645: 8 total, 8 up, 8 in 2024-09-13T13:42:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:57 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:42:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:58 smithi189 ceph-mon[32912]: osdmap e646: 8 total, 8 up, 8 in 2024-09-13T13:42:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:58 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:42:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:58 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:42:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:58 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:42:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:58 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[40255]: osdmap e646: 8 total, 8 up, 8 in 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[30106]: osdmap e646: 8 total, 8 up, 8 in 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:42:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:42:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:42:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:42:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:58 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:42:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:59 smithi189 ceph-mon[32912]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:59 smithi189 ceph-mon[32912]: osdmap e647: 8 total, 8 up, 8 in 2024-09-13T13:42:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:42:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[30106]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[30106]: osdmap e647: 8 total, 8 up, 8 in 2024-09-13T13:42:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[40255]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:42:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:42:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[40255]: osdmap e647: 8 total, 8 up, 8 in 2024-09-13T13:42:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:42:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:42:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:43:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:43:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:00 smithi189 ceph-mon[32912]: osdmap e648: 8 total, 8 up, 8 in 2024-09-13T13:43:00.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:43:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:00 smithi043 ceph-mon[40255]: osdmap e648: 8 total, 8 up, 8 in 2024-09-13T13:43:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:43:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:00 smithi043 ceph-mon[30106]: osdmap e648: 8 total, 8 up, 8 in 2024-09-13T13:43:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:01 smithi189 ceph-mon[32912]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:43:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:01 smithi189 ceph-mon[32912]: osdmap e649: 8 total, 8 up, 8 in 2024-09-13T13:43:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:01.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[40255]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:43:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[40255]: osdmap e649: 8 total, 8 up, 8 in 2024-09-13T13:43:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[30106]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:43:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[30106]: osdmap e649: 8 total, 8 up, 8 in 2024-09-13T13:43:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:02 smithi189 ceph-mon[32912]: osdmap e650: 8 total, 8 up, 8 in 2024-09-13T13:43:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:43:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:43:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:02 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:43:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[40255]: osdmap e650: 8 total, 8 up, 8 in 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[30106]: osdmap e650: 8 total, 8 up, 8 in 2024-09-13T13:43:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:43:02.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"} : dispatch 2024-09-13T13:43:02.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:02.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-132"}]': finished 2024-09-13T13:43:03.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:03 smithi043 ceph-mon[40255]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:43:03.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:03 smithi043 ceph-mon[40255]: osdmap e651: 8 total, 8 up, 8 in 2024-09-13T13:43:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:03 smithi043 ceph-mon[30106]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:43:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:03 smithi043 ceph-mon[30106]: osdmap e651: 8 total, 8 up, 8 in 2024-09-13T13:43:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:03 smithi189 ceph-mon[32912]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-13T13:43:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:03 smithi189 ceph-mon[32912]: osdmap e651: 8 total, 8 up, 8 in 2024-09-13T13:43:04.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:04 smithi189 ceph-mon[32912]: osdmap e652: 8 total, 8 up, 8 in 2024-09-13T13:43:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:04 smithi043 ceph-mon[30106]: osdmap e652: 8 total, 8 up, 8 in 2024-09-13T13:43:04.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:04 smithi043 ceph-mon[40255]: osdmap e652: 8 total, 8 up, 8 in 2024-09-13T13:43:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:43:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[30106]: pgmap v997: 172 pgs: 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[30106]: osdmap e653: 8 total, 8 up, 8 in 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[40255]: pgmap v997: 172 pgs: 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[40255]: osdmap e653: 8 total, 8 up, 8 in 2024-09-13T13:43:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:06.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:43:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:43:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:05 smithi189 ceph-mon[32912]: pgmap v997: 172 pgs: 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:05 smithi189 ceph-mon[32912]: osdmap e653: 8 total, 8 up, 8 in 2024-09-13T13:43:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:06.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[40255]: osdmap e654: 8 total, 8 up, 8 in 2024-09-13T13:43:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[30106]: osdmap e654: 8 total, 8 up, 8 in 2024-09-13T13:43:06.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:06.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:06.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:06 smithi189 ceph-mon[32912]: osdmap e654: 8 total, 8 up, 8 in 2024-09-13T13:43:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:07.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: pgmap v1000: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: osdmap e655: 8 total, 8 up, 8 in 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"}]': finished 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: osdmap e656: 8 total, 8 up, 8 in 2024-09-13T13:43:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"} : dispatch 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"} : dispatch 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: pgmap v1000: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: osdmap e655: 8 total, 8 up, 8 in 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"}]': finished 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: osdmap e656: 8 total, 8 up, 8 in 2024-09-13T13:43:07.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"} : dispatch 2024-09-13T13:43:07.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"} : dispatch 2024-09-13T13:43:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: pgmap v1000: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-13T13:43:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: osdmap e655: 8 total, 8 up, 8 in 2024-09-13T13:43:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-134"}]': finished 2024-09-13T13:43:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: osdmap e656: 8 total, 8 up, 8 in 2024-09-13T13:43:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"} : dispatch 2024-09-13T13:43:08.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"} : dispatch 2024-09-13T13:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"}]': finished 2024-09-13T13:43:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[30106]: osdmap e657: 8 total, 8 up, 8 in 2024-09-13T13:43:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"}]': finished 2024-09-13T13:43:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[40255]: osdmap e657: 8 total, 8 up, 8 in 2024-09-13T13:43:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:08.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:08 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-134", "mode": "writeback"}]': finished 2024-09-13T13:43:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:08 smithi189 ceph-mon[32912]: osdmap e657: 8 total, 8 up, 8 in 2024-09-13T13:43:09.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:08 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:09.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[30106]: pgmap v1004: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-13T13:43:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[30106]: osdmap e658: 8 total, 8 up, 8 in 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[40255]: pgmap v1004: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[40255]: osdmap e658: 8 total, 8 up, 8 in 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:09 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:09 smithi189 ceph-mon[32912]: pgmap v1004: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-13T13:43:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:09 smithi189 ceph-mon[32912]: osdmap e658: 8 total, 8 up, 8 in 2024-09-13T13:43:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:09 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"} : dispatch 2024-09-13T13:43:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:10 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:10 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"}]': finished 2024-09-13T13:43:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:10 smithi043 ceph-mon[30106]: osdmap e659: 8 total, 8 up, 8 in 2024-09-13T13:43:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:10 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:10 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"}]': finished 2024-09-13T13:43:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:10 smithi043 ceph-mon[40255]: osdmap e659: 8 total, 8 up, 8 in 2024-09-13T13:43:11.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:10 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:11.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:10 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-134"}]': finished 2024-09-13T13:43:11.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:10 smithi189 ceph-mon[32912]: osdmap e659: 8 total, 8 up, 8 in 2024-09-13T13:43:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:11 smithi043 ceph-mon[40255]: pgmap v1007: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-13T13:43:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:11 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:11 smithi043 ceph-mon[40255]: osdmap e660: 8 total, 8 up, 8 in 2024-09-13T13:43:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:11 smithi043 ceph-mon[30106]: pgmap v1007: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-13T13:43:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:11 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:11 smithi043 ceph-mon[30106]: osdmap e660: 8 total, 8 up, 8 in 2024-09-13T13:43:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:11 smithi189 ceph-mon[32912]: pgmap v1007: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-13T13:43:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:11 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:11 smithi189 ceph-mon[32912]: osdmap e660: 8 total, 8 up, 8 in 2024-09-13T13:43:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:13 smithi189 ceph-mon[32912]: osdmap e661: 8 total, 8 up, 8 in 2024-09-13T13:43:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:13.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:13 smithi189 ceph-mon[32912]: pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:43:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[30106]: osdmap e661: 8 total, 8 up, 8 in 2024-09-13T13:43:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[30106]: pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:43:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[40255]: osdmap e661: 8 total, 8 up, 8 in 2024-09-13T13:43:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:13 smithi043 ceph-mon[40255]: pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-13T13:43:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:14 smithi189 ceph-mon[32912]: osdmap e662: 8 total, 8 up, 8 in 2024-09-13T13:43:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:14 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:14.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:14 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:14.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:14.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[30106]: osdmap e662: 8 total, 8 up, 8 in 2024-09-13T13:43:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:14.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[40255]: osdmap e662: 8 total, 8 up, 8 in 2024-09-13T13:43:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:14.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:14 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:15 smithi189 ceph-mon[32912]: osdmap e663: 8 total, 8 up, 8 in 2024-09-13T13:43:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:15.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:15 smithi189 ceph-mon[32912]: pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[30106]: osdmap e663: 8 total, 8 up, 8 in 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[30106]: pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[40255]: osdmap e663: 8 total, 8 up, 8 in 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-mon[40255]: pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:43:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:43:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:43:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"}]': finished 2024-09-13T13:43:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:16 smithi189 ceph-mon[32912]: osdmap e664: 8 total, 8 up, 8 in 2024-09-13T13:43:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"} : dispatch 2024-09-13T13:43:16.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"} : dispatch 2024-09-13T13:43:16.149 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:43:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:43:16.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"}]': finished 2024-09-13T13:43:16.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[40255]: osdmap e664: 8 total, 8 up, 8 in 2024-09-13T13:43:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"} : dispatch 2024-09-13T13:43:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"} : dispatch 2024-09-13T13:43:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-136"}]': finished 2024-09-13T13:43:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[30106]: osdmap e664: 8 total, 8 up, 8 in 2024-09-13T13:43:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"} : dispatch 2024-09-13T13:43:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"} : dispatch 2024-09-13T13:43:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"}]': finished 2024-09-13T13:43:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: osdmap e665: 8 total, 8 up, 8 in 2024-09-13T13:43:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: pgmap v1016: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:17.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:17 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"}]': finished 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: osdmap e665: 8 total, 8 up, 8 in 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: pgmap v1016: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-136", "mode": "writeback"}]': finished 2024-09-13T13:43:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: osdmap e665: 8 total, 8 up, 8 in 2024-09-13T13:43:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: pgmap v1016: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:17.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:17 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:18 smithi189 ceph-mon[32912]: osdmap e666: 8 total, 8 up, 8 in 2024-09-13T13:43:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:18.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:18.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[30106]: osdmap e666: 8 total, 8 up, 8 in 2024-09-13T13:43:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:43:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[40255]: osdmap e666: 8 total, 8 up, 8 in 2024-09-13T13:43:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"} : dispatch 2024-09-13T13:43:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:19 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"}]': finished 2024-09-13T13:43:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:19 smithi189 ceph-mon[32912]: osdmap e667: 8 total, 8 up, 8 in 2024-09-13T13:43:19.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:19 smithi189 ceph-mon[32912]: pgmap v1019: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"}]': finished 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[30106]: osdmap e667: 8 total, 8 up, 8 in 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[30106]: pgmap v1019: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-136"}]': finished 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[40255]: osdmap e667: 8 total, 8 up, 8 in 2024-09-13T13:43:19.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:19 smithi043 ceph-mon[40255]: pgmap v1019: 204 pgs: 204 active+clean; 583 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:20.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:20 smithi189 ceph-mon[32912]: osdmap e668: 8 total, 8 up, 8 in 2024-09-13T13:43:20.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:20 smithi043 ceph-mon[30106]: osdmap e668: 8 total, 8 up, 8 in 2024-09-13T13:43:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:20 smithi043 ceph-mon[40255]: osdmap e668: 8 total, 8 up, 8 in 2024-09-13T13:43:21.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:21 smithi189 ceph-mon[32912]: osdmap e669: 8 total, 8 up, 8 in 2024-09-13T13:43:21.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:21.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:21 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:21.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:21 smithi189 ceph-mon[32912]: pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-13T13:43:21.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:21 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[40255]: osdmap e669: 8 total, 8 up, 8 in 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[40255]: pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[30106]: osdmap e669: 8 total, 8 up, 8 in 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[30106]: pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-13T13:43:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:21 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:22.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:22 smithi189 ceph-mon[32912]: osdmap e670: 8 total, 8 up, 8 in 2024-09-13T13:43:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:22.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:22 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:22.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:22.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[40255]: osdmap e670: 8 total, 8 up, 8 in 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[30106]: osdmap e670: 8 total, 8 up, 8 in 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:22.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:22 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:23 smithi189 ceph-mon[32912]: osdmap e671: 8 total, 8 up, 8 in 2024-09-13T13:43:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:23.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:23.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:23 smithi189 ceph-mon[32912]: pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-13T13:43:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[40255]: osdmap e671: 8 total, 8 up, 8 in 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[40255]: pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[30106]: osdmap e671: 8 total, 8 up, 8 in 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:23 smithi043 ceph-mon[30106]: pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-13T13:43:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:43:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:24 smithi189 ceph-mon[32912]: osdmap e672: 8 total, 8 up, 8 in 2024-09-13T13:43:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:24.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:24.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:43:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[40255]: osdmap e672: 8 total, 8 up, 8 in 2024-09-13T13:43:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:43:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[30106]: osdmap e672: 8 total, 8 up, 8 in 2024-09-13T13:43:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:43:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:25 smithi189 ceph-mon[32912]: osdmap e673: 8 total, 8 up, 8 in 2024-09-13T13:43:25.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:43:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:43:25.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:25 smithi189 ceph-mon[32912]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[40255]: osdmap e673: 8 total, 8 up, 8 in 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[40255]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[30106]: osdmap e673: 8 total, 8 up, 8 in 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-13T13:43:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-mon[30106]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 697 MiB used, 715 GiB / 715 GiB avail 2024-09-13T13:43:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:43:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:43:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:43:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:43:26.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:26 smithi189 ceph-mon[32912]: osdmap e674: 8 total, 8 up, 8 in 2024-09-13T13:43:26.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:43:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:43:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:43:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:26 smithi043 ceph-mon[30106]: osdmap e674: 8 total, 8 up, 8 in 2024-09-13T13:43:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-13T13:43:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:26 smithi043 ceph-mon[40255]: osdmap e674: 8 total, 8 up, 8 in 2024-09-13T13:43:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:27 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:27.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"} : dispatch 2024-09-13T13:43:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"} : dispatch 2024-09-13T13:43:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:27 smithi189 ceph-mon[32912]: pgmap v1030: 204 pgs: 204 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:27.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:27 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"} : dispatch 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"} : dispatch 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[30106]: pgmap v1030: 204 pgs: 204 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:27.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"} : dispatch 2024-09-13T13:43:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"} : dispatch 2024-09-13T13:43:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[40255]: pgmap v1030: 204 pgs: 204 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:27.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:27 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"}]': finished 2024-09-13T13:43:28.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:28 smithi189 ceph-mon[32912]: osdmap e675: 8 total, 8 up, 8 in 2024-09-13T13:43:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"}]': finished 2024-09-13T13:43:28.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:28 smithi043 ceph-mon[40255]: osdmap e675: 8 total, 8 up, 8 in 2024-09-13T13:43:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-138"}]': finished 2024-09-13T13:43:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:28 smithi043 ceph-mon[30106]: osdmap e675: 8 total, 8 up, 8 in 2024-09-13T13:43:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:29 smithi189 ceph-mon[32912]: osdmap e676: 8 total, 8 up, 8 in 2024-09-13T13:43:29.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:29 smithi189 ceph-mon[32912]: pgmap v1033: 172 pgs: 172 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:29 smithi043 ceph-mon[40255]: osdmap e676: 8 total, 8 up, 8 in 2024-09-13T13:43:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:29 smithi043 ceph-mon[40255]: pgmap v1033: 172 pgs: 172 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:29 smithi043 ceph-mon[30106]: osdmap e676: 8 total, 8 up, 8 in 2024-09-13T13:43:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:29 smithi043 ceph-mon[30106]: pgmap v1033: 172 pgs: 172 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:30.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:30 smithi043 ceph-mon[30106]: osdmap e677: 8 total, 8 up, 8 in 2024-09-13T13:43:30.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:30.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:30 smithi043 ceph-mon[40255]: osdmap e677: 8 total, 8 up, 8 in 2024-09-13T13:43:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:30.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:30 smithi189 ceph-mon[32912]: osdmap e677: 8 total, 8 up, 8 in 2024-09-13T13:43:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:30.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: osdmap e678: 8 total, 8 up, 8 in 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: osdmap e679: 8 total, 8 up, 8 in 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:43:31.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: osdmap e678: 8 total, 8 up, 8 in 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: osdmap e679: 8 total, 8 up, 8 in 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:43:31.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:43:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: osdmap e678: 8 total, 8 up, 8 in 2024-09-13T13:43:31.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: osdmap e679: 8 total, 8 up, 8 in 2024-09-13T13:43:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:43:31.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-13T13:43:33.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"}]': finished 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[30106]: osdmap e680: 8 total, 8 up, 8 in 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[30106]: pgmap v1039: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"}]': finished 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[40255]: osdmap e680: 8 total, 8 up, 8 in 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:43:33.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:33 smithi043 ceph-mon[40255]: pgmap v1039: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_count","val": "3"}]': finished 2024-09-13T13:43:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:33 smithi189 ceph-mon[32912]: osdmap e680: 8 total, 8 up, 8 in 2024-09-13T13:43:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:33 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:43:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:33 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-13T13:43:33.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:33 smithi189 ceph-mon[32912]: pgmap v1039: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"}]': finished 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[30106]: osdmap e681: 8 total, 8 up, 8 in 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"}]': finished 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[40255]: osdmap e681: 8 total, 8 up, 8 in 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:34.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:34 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_period","val": "3"}]': finished 2024-09-13T13:43:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:34 smithi189 ceph-mon[32912]: osdmap e681: 8 total, 8 up, 8 in 2024-09-13T13:43:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:34 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:34.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:34 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:43:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[30106]: osdmap e682: 8 total, 8 up, 8 in 2024-09-13T13:43:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:43:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:43:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[30106]: pgmap v1042: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:43:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:43:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[40255]: osdmap e682: 8 total, 8 up, 8 in 2024-09-13T13:43:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:43:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:43:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-mon[40255]: pgmap v1042: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:43:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:43:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:35 smithi189 ceph-mon[32912]: osdmap e682: 8 total, 8 up, 8 in 2024-09-13T13:43:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:35 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:43:35.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:35 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-13T13:43:35.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:35 smithi189 ceph-mon[32912]: pgmap v1042: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:43:35.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:43:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:43:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:43:36.172 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:43:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:43:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:36 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-13T13:43:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:36 smithi043 ceph-mon[30106]: osdmap e683: 8 total, 8 up, 8 in 2024-09-13T13:43:36.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:36 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:36 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-13T13:43:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:36 smithi043 ceph-mon[40255]: osdmap e683: 8 total, 8 up, 8 in 2024-09-13T13:43:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:36 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:36 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-13T13:43:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:36 smithi189 ceph-mon[32912]: osdmap e683: 8 total, 8 up, 8 in 2024-09-13T13:43:36.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:36 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:37 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:37 smithi043 ceph-mon[30106]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-13T13:43:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:37 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:37 smithi043 ceph-mon[40255]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-13T13:43:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:37 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:37.648 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:37 smithi189 ceph-mon[32912]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-13T13:43:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:39 smithi043 ceph-mon[30106]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-09-13T13:43:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:39 smithi043 ceph-mon[40255]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-09-13T13:43:39.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:39 smithi189 ceph-mon[32912]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-09-13T13:43:41.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:41 smithi189 ceph-mon[32912]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-09-13T13:43:41.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:41 smithi043 ceph-mon[30106]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-09-13T13:43:41.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:41 smithi043 ceph-mon[40255]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-09-13T13:43:43.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:43 smithi189 ceph-mon[32912]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.9 KiB/s wr, 2 op/s 2024-09-13T13:43:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:43 smithi043 ceph-mon[30106]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.9 KiB/s wr, 2 op/s 2024-09-13T13:43:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:43 smithi043 ceph-mon[40255]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.9 KiB/s wr, 2 op/s 2024-09-13T13:43:45.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:45 smithi189 ceph-mon[32912]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-13T13:43:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:45 smithi043 ceph-mon[30106]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-13T13:43:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:45 smithi043 ceph-mon[40255]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-13T13:43:45.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:43:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:43:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:43:46.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:43:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:43:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:47 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:47 smithi189 ceph-mon[32912]: pgmap v1049: 204 pgs: 204 active+clean; 4.5 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-09-13T13:43:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"} : dispatch 2024-09-13T13:43:47.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"} : dispatch 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[40255]: pgmap v1049: 204 pgs: 204 active+clean; 4.5 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"} : dispatch 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"} : dispatch 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[30106]: pgmap v1049: 204 pgs: 204 active+clean; 4.5 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-09-13T13:43:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:43:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"} : dispatch 2024-09-13T13:43:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"} : dispatch 2024-09-13T13:43:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:48 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"}]': finished 2024-09-13T13:43:48.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:48 smithi189 ceph-mon[32912]: osdmap e684: 8 total, 8 up, 8 in 2024-09-13T13:43:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:48 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"}]': finished 2024-09-13T13:43:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:48 smithi043 ceph-mon[30106]: osdmap e684: 8 total, 8 up, 8 in 2024-09-13T13:43:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:48 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-140"}]': finished 2024-09-13T13:43:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:48 smithi043 ceph-mon[40255]: osdmap e684: 8 total, 8 up, 8 in 2024-09-13T13:43:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:49 smithi189 ceph-mon[32912]: pgmap v1051: 204 pgs: 204 active+clean; 4.5 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-13T13:43:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:49 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:49 smithi189 ceph-mon[32912]: osdmap e685: 8 total, 8 up, 8 in 2024-09-13T13:43:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:49 smithi043 ceph-mon[30106]: pgmap v1051: 204 pgs: 204 active+clean; 4.5 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-13T13:43:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:49 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:49 smithi043 ceph-mon[30106]: osdmap e685: 8 total, 8 up, 8 in 2024-09-13T13:43:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:49 smithi043 ceph-mon[40255]: pgmap v1051: 204 pgs: 204 active+clean; 4.5 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-13T13:43:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:49 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:49 smithi043 ceph-mon[40255]: osdmap e685: 8 total, 8 up, 8 in 2024-09-13T13:43:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:50 smithi189 ceph-mon[32912]: osdmap e686: 8 total, 8 up, 8 in 2024-09-13T13:43:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:50.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:50.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[30106]: osdmap e686: 8 total, 8 up, 8 in 2024-09-13T13:43:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[40255]: osdmap e686: 8 total, 8 up, 8 in 2024-09-13T13:43:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:43:50.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:43:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:51 smithi189 ceph-mon[32912]: pgmap v1054: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:43:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:51 smithi189 ceph-mon[32912]: osdmap e687: 8 total, 8 up, 8 in 2024-09-13T13:43:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:51 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:51.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:51 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[30106]: pgmap v1054: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[30106]: osdmap e687: 8 total, 8 up, 8 in 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[40255]: pgmap v1054: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[40255]: osdmap e687: 8 total, 8 up, 8 in 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:51 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:43:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:52 smithi189 ceph-mon[32912]: osdmap e688: 8 total, 8 up, 8 in 2024-09-13T13:43:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:43:52.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:43:52.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[30106]: osdmap e688: 8 total, 8 up, 8 in 2024-09-13T13:43:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:43:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:43:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:43:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[40255]: osdmap e688: 8 total, 8 up, 8 in 2024-09-13T13:43:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:43:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:43:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:53 smithi189 ceph-mon[32912]: pgmap v1057: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"}]': finished 2024-09-13T13:43:53.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:53 smithi189 ceph-mon[32912]: osdmap e689: 8 total, 8 up, 8 in 2024-09-13T13:43:53.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"} : dispatch 2024-09-13T13:43:53.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"} : dispatch 2024-09-13T13:43:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[40255]: pgmap v1057: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"}]': finished 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[40255]: osdmap e689: 8 total, 8 up, 8 in 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"} : dispatch 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"} : dispatch 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[30106]: pgmap v1057: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-142"}]': finished 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[30106]: osdmap e689: 8 total, 8 up, 8 in 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"} : dispatch 2024-09-13T13:43:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"} : dispatch 2024-09-13T13:43:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:54 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:54.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"}]': finished 2024-09-13T13:43:54.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:54 smithi189 ceph-mon[32912]: osdmap e690: 8 total, 8 up, 8 in 2024-09-13T13:43:54.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:54.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:54.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:54 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:43:54.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:54 smithi189 ceph-mon[32912]: osdmap e691: 8 total, 8 up, 8 in 2024-09-13T13:43:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"}]': finished 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[30106]: osdmap e690: 8 total, 8 up, 8 in 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[30106]: osdmap e691: 8 total, 8 up, 8 in 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-142", "mode": "writeback"}]': finished 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[40255]: osdmap e690: 8 total, 8 up, 8 in 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:43:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:43:54.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:54 smithi043 ceph-mon[40255]: osdmap e691: 8 total, 8 up, 8 in 2024-09-13T13:43:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:55 smithi189 ceph-mon[32912]: pgmap v1060: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:43:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:55.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:55 smithi043 ceph-mon[30106]: pgmap v1060: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:43:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:55.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:43:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:43:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:43:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:55 smithi043 ceph-mon[40255]: pgmap v1060: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:43:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:43:56.399 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:43:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:43:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:43:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:56 smithi189 ceph-mon[32912]: osdmap e692: 8 total, 8 up, 8 in 2024-09-13T13:43:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:56.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:43:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[30106]: osdmap e692: 8 total, 8 up, 8 in 2024-09-13T13:43:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:43:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[40255]: osdmap e692: 8 total, 8 up, 8 in 2024-09-13T13:43:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:43:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:43:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:43:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: osdmap e693: 8 total, 8 up, 8 in 2024-09-13T13:43:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:43:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:43:57.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:57 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:57.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:57.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: osdmap e693: 8 total, 8 up, 8 in 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:43:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:43:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: osdmap e693: 8 total, 8 up, 8 in 2024-09-13T13:43:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:43:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:43:57.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:57 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:43:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:43:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:58 smithi189 ceph-mon[32912]: osdmap e694: 8 total, 8 up, 8 in 2024-09-13T13:43:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:43:58.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:43:58.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:58 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:43:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:43:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[40255]: osdmap e694: 8 total, 8 up, 8 in 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[30106]: osdmap e694: 8 total, 8 up, 8 in 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-13T13:43:58.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:58 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:43:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:43:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-13T13:43:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: osdmap e695: 8 total, 8 up, 8 in 2024-09-13T13:43:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:43:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:43:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:43:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:43:59.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:43:59 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:43:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: osdmap e695: 8 total, 8 up, 8 in 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-13T13:43:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-13T13:43:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: osdmap e695: 8 total, 8 up, 8 in 2024-09-13T13:43:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:43:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-13T13:43:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:43:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:43:59.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:43:59 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:44:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-13T13:44:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:00 smithi189 ceph-mon[32912]: osdmap e696: 8 total, 8 up, 8 in 2024-09-13T13:44:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-13T13:44:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[30106]: osdmap e696: 8 total, 8 up, 8 in 2024-09-13T13:44:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-13T13:44:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[40255]: osdmap e696: 8 total, 8 up, 8 in 2024-09-13T13:44:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:01 smithi189 ceph-mon[32912]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:44:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:01 smithi189 ceph-mon[32912]: osdmap e697: 8 total, 8 up, 8 in 2024-09-13T13:44:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:01.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:01 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[30106]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[30106]: osdmap e697: 8 total, 8 up, 8 in 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[40255]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[40255]: osdmap e697: 8 total, 8 up, 8 in 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:01 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"}]': finished 2024-09-13T13:44:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:02 smithi189 ceph-mon[32912]: osdmap e698: 8 total, 8 up, 8 in 2024-09-13T13:44:02.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:02.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:02 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"}]': finished 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[30106]: osdmap e698: 8 total, 8 up, 8 in 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"}]': finished 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[40255]: osdmap e698: 8 total, 8 up, 8 in 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:02.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:02 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-142"} : dispatch 2024-09-13T13:44:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[30106]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:44:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[30106]: osdmap e699: 8 total, 8 up, 8 in 2024-09-13T13:44:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[30106]: osdmap e700: 8 total, 8 up, 8 in 2024-09-13T13:44:03.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:03.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[40255]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:44:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[40255]: osdmap e699: 8 total, 8 up, 8 in 2024-09-13T13:44:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[40255]: osdmap e700: 8 total, 8 up, 8 in 2024-09-13T13:44:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:03.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:03 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:03 smithi189 ceph-mon[32912]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-13T13:44:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:03 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:03 smithi189 ceph-mon[32912]: osdmap e699: 8 total, 8 up, 8 in 2024-09-13T13:44:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:03 smithi189 ceph-mon[32912]: osdmap e700: 8 total, 8 up, 8 in 2024-09-13T13:44:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:03 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:05 smithi189 ceph-mon[32912]: pgmap v1075: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:05 smithi189 ceph-mon[32912]: osdmap e701: 8 total, 8 up, 8 in 2024-09-13T13:44:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:05 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:05.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:05.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:44:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:44:06.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[30106]: pgmap v1075: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:06.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:06.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[30106]: osdmap e701: 8 total, 8 up, 8 in 2024-09-13T13:44:06.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:06.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:06.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:06.044 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[40255]: pgmap v1075: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:06.044 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:06.044 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[40255]: osdmap e701: 8 total, 8 up, 8 in 2024-09-13T13:44:06.044 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:06.044 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:06.044 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:06.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:44:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:44:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:06 smithi189 ceph-mon[32912]: osdmap e702: 8 total, 8 up, 8 in 2024-09-13T13:44:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:06 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:06 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[30106]: osdmap e702: 8 total, 8 up, 8 in 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[40255]: osdmap e702: 8 total, 8 up, 8 in 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:06 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:07 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:07 smithi189 ceph-mon[32912]: pgmap v1078: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"}]': finished 2024-09-13T13:44:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:07 smithi189 ceph-mon[32912]: osdmap e703: 8 total, 8 up, 8 in 2024-09-13T13:44:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:07 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"} : dispatch 2024-09-13T13:44:07.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:07 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"} : dispatch 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[30106]: pgmap v1078: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"}]': finished 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[30106]: osdmap e703: 8 total, 8 up, 8 in 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"} : dispatch 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"} : dispatch 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[40255]: pgmap v1078: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:07.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-144"}]': finished 2024-09-13T13:44:07.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[40255]: osdmap e703: 8 total, 8 up, 8 in 2024-09-13T13:44:07.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"} : dispatch 2024-09-13T13:44:07.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:07 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"} : dispatch 2024-09-13T13:44:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:08 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:44:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:08 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"}]': finished 2024-09-13T13:44:08.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:08 smithi189 ceph-mon[32912]: osdmap e704: 8 total, 8 up, 8 in 2024-09-13T13:44:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:08 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:44:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:08 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"}]': finished 2024-09-13T13:44:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:08 smithi043 ceph-mon[30106]: osdmap e704: 8 total, 8 up, 8 in 2024-09-13T13:44:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:08 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:44:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:08 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-144", "mode": "readproxy"}]': finished 2024-09-13T13:44:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:08 smithi043 ceph-mon[40255]: osdmap e704: 8 total, 8 up, 8 in 2024-09-13T13:44:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:09 smithi189 ceph-mon[32912]: pgmap v1081: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:09 smithi043 ceph-mon[30106]: pgmap v1081: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:09 smithi043 ceph-mon[40255]: pgmap v1081: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:11 smithi189 ceph-mon[32912]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 174 B/s wr, 2 op/s 2024-09-13T13:44:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:11 smithi043 ceph-mon[30106]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 174 B/s wr, 2 op/s 2024-09-13T13:44:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:11 smithi043 ceph-mon[40255]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 174 B/s wr, 2 op/s 2024-09-13T13:44:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:12 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:12 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:12 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:13 smithi189 ceph-mon[32912]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 897 B/s rd, 0 op/s 2024-09-13T13:44:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:13 smithi043 ceph-mon[30106]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 897 B/s rd, 0 op/s 2024-09-13T13:44:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:13 smithi043 ceph-mon[40255]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 897 B/s rd, 0 op/s 2024-09-13T13:44:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:15 smithi189 ceph-mon[32912]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-13T13:44:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:15 smithi043 ceph-mon[30106]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-13T13:44:15.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:44:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:44:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:44:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:15 smithi043 ceph-mon[40255]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-13T13:44:16.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:44:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:44:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:17 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:17 smithi189 ceph-mon[32912]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:44:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:17 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:17 smithi043 ceph-mon[30106]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:44:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:17 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:17 smithi043 ceph-mon[40255]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-13T13:44:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:18.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[30106]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[30106]: osdmap e705: 8 total, 8 up, 8 in 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[40255]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[40255]: osdmap e705: 8 total, 8 up, 8 in 2024-09-13T13:44:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:19.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:19 smithi189 ceph-mon[32912]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:44:20.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:20.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:19 smithi189 ceph-mon[32912]: osdmap e705: 8 total, 8 up, 8 in 2024-09-13T13:44:20.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:20.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:44:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"}]': finished 2024-09-13T13:44:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: osdmap e706: 8 total, 8 up, 8 in 2024-09-13T13:44:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:20.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:20.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:20.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:20.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:44:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"}]': finished 2024-09-13T13:44:20.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: osdmap e706: 8 total, 8 up, 8 in 2024-09-13T13:44:20.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:20.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:20.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:20.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:20.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:21.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:44:21.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"}]': finished 2024-09-13T13:44:21.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: osdmap e706: 8 total, 8 up, 8 in 2024-09-13T13:44:21.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-144"} : dispatch 2024-09-13T13:44:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:21 smithi043 ceph-mon[30106]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:44:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:21 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:21 smithi043 ceph-mon[30106]: osdmap e707: 8 total, 8 up, 8 in 2024-09-13T13:44:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:21 smithi043 ceph-mon[40255]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:44:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:21 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:21 smithi043 ceph-mon[40255]: osdmap e707: 8 total, 8 up, 8 in 2024-09-13T13:44:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:21 smithi189 ceph-mon[32912]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:44:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:21 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:21 smithi189 ceph-mon[32912]: osdmap e707: 8 total, 8 up, 8 in 2024-09-13T13:44:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:22 smithi043 ceph-mon[30106]: osdmap e708: 8 total, 8 up, 8 in 2024-09-13T13:44:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:22 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:22 smithi043 ceph-mon[40255]: osdmap e708: 8 total, 8 up, 8 in 2024-09-13T13:44:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:22 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:22 smithi189 ceph-mon[32912]: osdmap e708: 8 total, 8 up, 8 in 2024-09-13T13:44:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:22 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[30106]: pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[30106]: osdmap e709: 8 total, 8 up, 8 in 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[40255]: pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[40255]: osdmap e709: 8 total, 8 up, 8 in 2024-09-13T13:44:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:23.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:23 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:24.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:23 smithi189 ceph-mon[32912]: pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:44:24.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:24.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:23 smithi189 ceph-mon[32912]: osdmap e709: 8 total, 8 up, 8 in 2024-09-13T13:44:24.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:23 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:24.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:23 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[30106]: osdmap e710: 8 total, 8 up, 8 in 2024-09-13T13:44:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[40255]: osdmap e710: 8 total, 8 up, 8 in 2024-09-13T13:44:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:24 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:24 smithi189 ceph-mon[32912]: osdmap e710: 8 total, 8 up, 8 in 2024-09-13T13:44:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:24 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:25.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:24 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:25.951 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:25 smithi189 ceph-mon[32912]: pgmap v1095: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:25.952 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"}]': finished 2024-09-13T13:44:25.952 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:25 smithi189 ceph-mon[32912]: osdmap e711: 8 total, 8 up, 8 in 2024-09-13T13:44:25.952 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:25 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"} : dispatch 2024-09-13T13:44:25.952 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:25 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"} : dispatch 2024-09-13T13:44:25.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:44:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[30106]: pgmap v1095: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"}]': finished 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[30106]: osdmap e711: 8 total, 8 up, 8 in 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"} : dispatch 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"} : dispatch 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[40255]: pgmap v1095: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-76627-111", "overlaypool": "test-rados-api-smithi043-76627-146"}]': finished 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[40255]: osdmap e711: 8 total, 8 up, 8 in 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"} : dispatch 2024-09-13T13:44:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:25 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"} : dispatch 2024-09-13T13:44:26.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:44:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:44:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:44:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"}]': finished 2024-09-13T13:44:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[30106]: osdmap e712: 8 total, 8 up, 8 in 2024-09-13T13:44:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:44:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:44:26.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:44:26.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"}]': finished 2024-09-13T13:44:26.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[40255]: osdmap e712: 8 total, 8 up, 8 in 2024-09-13T13:44:26.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:44:26.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:26 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:44:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:26 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-13T13:44:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-76627-146", "mode": "writeback"}]': finished 2024-09-13T13:44:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:26 smithi189 ceph-mon[32912]: osdmap e712: 8 total, 8 up, 8 in 2024-09-13T13:44:27.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:26 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:44:27.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:26 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-13T13:44:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[30106]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[30106]: osdmap e713: 8 total, 8 up, 8 in 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[40255]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:44:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:44:27.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[40255]: osdmap e713: 8 total, 8 up, 8 in 2024-09-13T13:44:27.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:44:27.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:44:27.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:27 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:27 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:27 smithi189 ceph-mon[32912]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:44:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_count","val": "2"}]': finished 2024-09-13T13:44:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:27 smithi189 ceph-mon[32912]: osdmap e713: 8 total, 8 up, 8 in 2024-09-13T13:44:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:27 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:44:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:27 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-13T13:44:28.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:27 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[30106]: osdmap e714: 8 total, 8 up, 8 in 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[40255]: osdmap e714: 8 total, 8 up, 8 in 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:44:28.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:28 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:44:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_period","val": "600"}]': finished 2024-09-13T13:44:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:28 smithi189 ceph-mon[32912]: osdmap e714: 8 total, 8 up, 8 in 2024-09-13T13:44:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:28 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:44:29.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:28 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-13T13:44:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[30106]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[30106]: osdmap e715: 8 total, 8 up, 8 in 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[40255]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[40255]: osdmap e715: 8 total, 8 up, 8 in 2024-09-13T13:44:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:44:29.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:29 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:44:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:29 smithi189 ceph-mon[32912]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-13T13:44:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:29 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-13T13:44:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-13T13:44:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:29 smithi189 ceph-mon[32912]: osdmap e715: 8 total, 8 up, 8 in 2024-09-13T13:44:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:29 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:44:30.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:29 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-13T13:44:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:44:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[30106]: osdmap e716: 8 total, 8 up, 8 in 2024-09-13T13:44:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:44:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:44:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:44:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[40255]: osdmap e716: 8 total, 8 up, 8 in 2024-09-13T13:44:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:44:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:30 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:44:31.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-13T13:44:31.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:30 smithi189 ceph-mon[32912]: osdmap e716: 8 total, 8 up, 8 in 2024-09-13T13:44:31.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:30 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:44:31.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:30 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-13T13:44:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:31 smithi043 ceph-mon[30106]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-13T13:44:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:31 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"}]': finished 2024-09-13T13:44:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:31 smithi043 ceph-mon[30106]: osdmap e717: 8 total, 8 up, 8 in 2024-09-13T13:44:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:31 smithi043 ceph-mon[40255]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-13T13:44:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:31 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"}]': finished 2024-09-13T13:44:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:31 smithi043 ceph-mon[40255]: osdmap e717: 8 total, 8 up, 8 in 2024-09-13T13:44:32.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:31 smithi189 ceph-mon[32912]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-13T13:44:32.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:31 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-146","var": "target_max_objects","val": "1"}]': finished 2024-09-13T13:44:32.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:31 smithi189 ceph-mon[32912]: osdmap e717: 8 total, 8 up, 8 in 2024-09-13T13:44:33.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:32 smithi189 ceph-mon[32912]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-13T13:44:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:32 smithi043 ceph-mon[30106]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-13T13:44:33.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:32 smithi043 ceph-mon[40255]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-13T13:44:34.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:33 smithi043 ceph-mon[30106]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-13T13:44:34.143 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:33 smithi043 ceph-mon[40255]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-13T13:44:34.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:33 smithi189 ceph-mon[32912]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-13T13:44:35.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:44:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:44:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:44:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:35 smithi043 ceph-mon[30106]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:35 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:44:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:35 smithi043 ceph-mon[40255]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:35 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:44:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:35 smithi189 ceph-mon[32912]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:36.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:35 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:44:36.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:36.149 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:44:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:44:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:37 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:37 smithi189 ceph-mon[32912]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:38.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:37 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:38.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:37 smithi043 ceph-mon[30106]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:38.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:37 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:38.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:37 smithi043 ceph-mon[40255]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:39 smithi189 ceph-mon[32912]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:40.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:39 smithi043 ceph-mon[30106]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:39 smithi043 ceph-mon[40255]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-13T13:44:41.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:41.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:40 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:40 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:40 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:41 smithi189 ceph-mon[32912]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:44:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:41 smithi189 ceph-mon[32912]: osdmap e718: 8 total, 8 up, 8 in 2024-09-13T13:44:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:41 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:41 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:42.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[30106]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:44:42.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[30106]: osdmap e718: 8 total, 8 up, 8 in 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[40255]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[40255]: osdmap e718: 8 total, 8 up, 8 in 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:41 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"}]': finished 2024-09-13T13:44:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:42 smithi189 ceph-mon[32912]: osdmap e719: 8 total, 8 up, 8 in 2024-09-13T13:44:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:42 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:43.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:42 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"}]': finished 2024-09-13T13:44:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[30106]: osdmap e719: 8 total, 8 up, 8 in 2024-09-13T13:44:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"}]': finished 2024-09-13T13:44:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[40255]: osdmap e719: 8 total, 8 up, 8 in 2024-09-13T13:44:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:42 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-146"} : dispatch 2024-09-13T13:44:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:43 smithi189 ceph-mon[32912]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:44:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:43 smithi189 ceph-mon[32912]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:43 smithi189 ceph-mon[32912]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-13T13:44:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:43 smithi189 ceph-mon[32912]: osdmap e720: 8 total, 8 up, 8 in 2024-09-13T13:44:44.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[30106]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:44:44.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[30106]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:44.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[30106]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-13T13:44:44.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[30106]: osdmap e720: 8 total, 8 up, 8 in 2024-09-13T13:44:44.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[40255]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-13T13:44:44.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[40255]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:44.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[40255]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-13T13:44:44.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:43 smithi043 ceph-mon[40255]: osdmap e720: 8 total, 8 up, 8 in 2024-09-13T13:44:45.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:44 smithi189 ceph-mon[32912]: osdmap e721: 8 total, 8 up, 8 in 2024-09-13T13:44:45.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:44 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:45.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:44 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:45.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:44 smithi043 ceph-mon[30106]: osdmap e721: 8 total, 8 up, 8 in 2024-09-13T13:44:45.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:44 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:45.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:44 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:44 smithi043 ceph-mon[40255]: osdmap e721: 8 total, 8 up, 8 in 2024-09-13T13:44:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:44 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:45.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:44 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:45.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:44:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:44:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[30106]: pgmap v1116: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[30106]: osdmap e722: 8 total, 8 up, 8 in 2024-09-13T13:44:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[40255]: pgmap v1116: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[40255]: osdmap e722: 8 total, 8 up, 8 in 2024-09-13T13:44:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:45.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:45 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:45 smithi189 ceph-mon[32912]: pgmap v1116: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:45 smithi189 ceph-mon[32912]: osdmap e722: 8 total, 8 up, 8 in 2024-09-13T13:44:46.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:45 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:46.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:45 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-13T13:44:46.149 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:44:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:44:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:46 smithi189 ceph-mon[32912]: osdmap e723: 8 total, 8 up, 8 in 2024-09-13T13:44:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:46 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:47.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:46 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:47.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[40255]: osdmap e723: 8 total, 8 up, 8 in 2024-09-13T13:44:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:47.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-13T13:44:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[30106]: osdmap e723: 8 total, 8 up, 8 in 2024-09-13T13:44:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:47.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:46 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: pgmap v1119: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-13T13:44:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"}]': finished 2024-09-13T13:44:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: osdmap e724: 8 total, 8 up, 8 in 2024-09-13T13:44:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:48.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:47 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:48.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: pgmap v1119: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"}]': finished 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: osdmap e724: 8 total, 8 up, 8 in 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: pgmap v1119: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"}]': finished 2024-09-13T13:44:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: osdmap e724: 8 total, 8 up, 8 in 2024-09-13T13:44:48.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:48.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:48.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:48.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:47 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-148"} : dispatch 2024-09-13T13:44:49.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:48 smithi189 ceph-mon[32912]: osdmap e725: 8 total, 8 up, 8 in 2024-09-13T13:44:49.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:48 smithi043 ceph-mon[30106]: osdmap e725: 8 total, 8 up, 8 in 2024-09-13T13:44:49.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:48 smithi043 ceph-mon[40255]: osdmap e725: 8 total, 8 up, 8 in 2024-09-13T13:44:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:49 smithi189 ceph-mon[32912]: pgmap v1122: 172 pgs: 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:49 smithi189 ceph-mon[32912]: osdmap e726: 8 total, 8 up, 8 in 2024-09-13T13:44:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:49 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:49 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:50.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[30106]: pgmap v1122: 172 pgs: 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[30106]: osdmap e726: 8 total, 8 up, 8 in 2024-09-13T13:44:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[40255]: pgmap v1122: 172 pgs: 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-13T13:44:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[40255]: osdmap e726: 8 total, 8 up, 8 in 2024-09-13T13:44:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:49 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:51.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:51.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: osdmap e727: 8 total, 8 up, 8 in 2024-09-13T13:44:51.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:51.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:51.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-150"} : dispatch 2024-09-13T13:44:51.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-150"} : dispatch 2024-09-13T13:44:51.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:44:51.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: osdmap e727: 8 total, 8 up, 8 in 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-150"} : dispatch 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-150"} : dispatch 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: osdmap e727: 8 total, 8 up, 8 in 2024-09-13T13:44:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-150"} : dispatch 2024-09-13T13:44:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-150"} : dispatch 2024-09-13T13:44:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:44:51.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:44:52.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:51 smithi189 ceph-mon[32912]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-13T13:44:52.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:51 smithi189 ceph-mon[32912]: osdmap e728: 8 total, 8 up, 8 in 2024-09-13T13:44:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:51 smithi043 ceph-mon[30106]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-13T13:44:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:51 smithi043 ceph-mon[30106]: osdmap e728: 8 total, 8 up, 8 in 2024-09-13T13:44:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:51 smithi043 ceph-mon[40255]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-13T13:44:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:51 smithi043 ceph-mon[40255]: osdmap e728: 8 total, 8 up, 8 in 2024-09-13T13:44:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:52 smithi189 ceph-mon[32912]: osdmap e729: 8 total, 8 up, 8 in 2024-09-13T13:44:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:52 smithi189 ceph-mon[32912]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[30106]: osdmap e729: 8 total, 8 up, 8 in 2024-09-13T13:44:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[30106]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[40255]: osdmap e729: 8 total, 8 up, 8 in 2024-09-13T13:44:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:52 smithi043 ceph-mon[40255]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:44:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:53 smithi189 ceph-mon[32912]: pgmap v1128: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:44:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:54.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:53 smithi189 ceph-mon[32912]: osdmap e730: 8 total, 8 up, 8 in 2024-09-13T13:44:54.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:54.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:54.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-152"} : dispatch 2024-09-13T13:44:54.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-152"} : dispatch 2024-09-13T13:44:54.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[30106]: pgmap v1128: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:44:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[30106]: osdmap e730: 8 total, 8 up, 8 in 2024-09-13T13:44:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-152"} : dispatch 2024-09-13T13:44:54.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-152"} : dispatch 2024-09-13T13:44:54.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[40255]: pgmap v1128: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-13T13:44:54.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:54.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[40255]: osdmap e730: 8 total, 8 up, 8 in 2024-09-13T13:44:54.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:54.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:54.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-152"} : dispatch 2024-09-13T13:44:54.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-152"} : dispatch 2024-09-13T13:44:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:54 smithi043 ceph-mon[30106]: osdmap e731: 8 total, 8 up, 8 in 2024-09-13T13:44:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:54 smithi043 ceph-mon[40255]: osdmap e731: 8 total, 8 up, 8 in 2024-09-13T13:44:55.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:54 smithi189 ceph-mon[32912]: osdmap e731: 8 total, 8 up, 8 in 2024-09-13T13:44:55.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:44:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:44:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[40255]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[40255]: osdmap e732: 8 total, 8 up, 8 in 2024-09-13T13:44:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[30106]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[30106]: osdmap e732: 8 total, 8 up, 8 in 2024-09-13T13:44:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:55.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:55 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:56.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:44:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:44:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:55 smithi189 ceph-mon[32912]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:55 smithi189 ceph-mon[32912]: osdmap e732: 8 total, 8 up, 8 in 2024-09-13T13:44:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:55 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:56.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:55 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T13:44:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: osdmap e733: 8 total, 8 up, 8 in 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-111","var": "dedup_tier","val": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-111","var": "dedup_tier","val": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:57.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[30106]: pgmap v1134: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: osdmap e733: 8 total, 8 up, 8 in 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-111","var": "dedup_tier","val": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-111","var": "dedup_tier","val": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:57.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:56 smithi043 ceph-mon[40255]: pgmap v1134: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-13T13:44:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-76627-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T13:44:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: osdmap e733: 8 total, 8 up, 8 in 2024-09-13T13:44:57.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-111","var": "dedup_tier","val": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi043-76627-111","var": "dedup_tier","val": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-76627-111", "tierpool": "test-rados-api-smithi043-76627-154"} : dispatch 2024-09-13T13:44:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:44:57.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:56 smithi189 ceph-mon[32912]: pgmap v1134: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-13T13:44:58.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:57 smithi043 ceph-mon[30106]: osdmap e734: 8 total, 8 up, 8 in 2024-09-13T13:44:58.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:57 smithi043 ceph-mon[40255]: osdmap e734: 8 total, 8 up, 8 in 2024-09-13T13:44:58.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:57 smithi189 ceph-mon[32912]: osdmap e734: 8 total, 8 up, 8 in 2024-09-13T13:44:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[30106]: osdmap e735: 8 total, 8 up, 8 in 2024-09-13T13:44:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:59.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[30106]: pgmap v1137: 164 pgs: 164 active+clean; 583 KiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[40255]: osdmap e735: 8 total, 8 up, 8 in 2024-09-13T13:44:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:58 smithi043 ceph-mon[40255]: pgmap v1137: 164 pgs: 164 active+clean; 583 KiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:58 smithi189 ceph-mon[32912]: osdmap e735: 8 total, 8 up, 8 in 2024-09-13T13:44:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:58 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:58 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:44:59.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:58 smithi189 ceph-mon[32912]: pgmap v1137: 164 pgs: 164 active+clean; 583 KiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:44:59.941 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (8040 ms) 2024-09-13T13:44:59.941 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12451 ms) 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8056 ms) 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13093 ms) 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7382 ms) 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8052 ms) 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-13T13:44:59.942 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9061 ms) 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1726235016 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,1726235021,1726235022,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,1726235021,1726235022,0 2024-09-13T13:44:59.943 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,1726235021,1726235022,0 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,1726235021,1726235022,1726235024,1726235025,0 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,1726235021,1726235022,1726235024,1726235025,0 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235016,1726235018,1726235019,1726235021,1726235022,1726235024,1726235025,0 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1726235019,1726235021,1726235022,1726235024,1726235025,1726235027,1726235028,0 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1726235019, trimmed 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20383 ms) 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-13T13:44:59.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14068 ms) 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18070 ms) 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22148 ms) 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5046 ms) 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3021 ms) 2024-09-13T13:44:59.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3023 ms) 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3023 ms) 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (229703 ms total) 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (843252 ms total) 2024-09-13T13:44:59.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-13T13:44:59.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76284 2024-09-13T13:44:59.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76284 2024-09-13T13:44:59.952 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76534 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76534 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77074 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77074 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76740 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76740 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76654 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76654 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.953 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76360 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76360 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76695 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76695 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77152 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77152 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77160 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77160 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76385 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76385 2024-09-13T13:44:59.954 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77102 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77102 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76259 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76259 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76322 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76322 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76960 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76960 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.955 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76250 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76250 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76270 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76270 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77361 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77361 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76602 2024-09-13T13:44:59.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76602 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77495 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77495 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76680 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76680 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77142 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77142 2024-09-13T13:44:59.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77606 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77606 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76616 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76616 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76985 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76985 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77313 2024-09-13T13:44:59.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77313 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77130 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77130 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77453 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77453 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76932 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76932 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76498 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76498 2024-09-13T13:44:59.959 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77529 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77529 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76759 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76759 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76243 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76243 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76301 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76301 2024-09-13T13:44:59.960 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76726 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76726 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76340 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76340 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76864 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76864 2024-09-13T13:44:59.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=76894 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 76894 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77257 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77257 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=77659 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 77659 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ exit 0 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ cleanup 2024-09-13T13:44:59.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ pkill -P 76237 2024-09-13T13:44:59.963 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-09-13T13:44:59.964 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T13:44:59.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T13:44:59.996 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-13T13:44:59.998 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-13T13:44:59.998 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b0201b8c79733293453e7f10a10c7fa43119222b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-13T13:45:00.054 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-09-13T13:45:00.059 INFO:tasks.workunit.client.0.smithi043.stderr:+ p=002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:45:00.059 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create 002fcfa4-dfdd-44b5-b965-f404930ab85a 12 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 -- 172.21.15.43:0/3545519337 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 msgr2=0x7fe3c410f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/3545519337 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 0x7fe3c410f4d0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009be0 tx=0x7fe3bc02a6c0 comp rx=0 tx=0).stop 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 -- 172.21.15.43:0/3545519337 shutdown_connections 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/3545519337 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 0x7fe3c410f4d0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/3545519337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4104520 0x7fe3c410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/3545519337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c4103c00 0x7fe3c4103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 -- 172.21.15.43:0/3545519337 >> 172.21.15.43:0/3545519337 conn(0x7fe3c40fd610 msgr2=0x7fe3c40ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 -- 172.21.15.43:0/3545519337 shutdown_connections 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.143+0000 7fe3cbc72640 1 -- 172.21.15.43:0/3545519337 wait complete. 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 Processor -- start 2024-09-13T13:45:00.145 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 -- start start 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4103c00 0x7fe3c41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c4104520 0x7fe3c4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 0x7fe3c419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41121b0 con 0x7fe3c4103c00 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4112030 con 0x7fe3c4104520 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3cbc72640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c4112330 con 0x7fe3c410d050 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4103c00 0x7fe3c41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4103c00 0x7fe3c41987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42258/0 (socket says 172.21.15.43:42258) 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3ca20f640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 0x7fe3c419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 -- 172.21.15.43:0/4209463749 learned_addr learned my addr 172.21.15.43:0/4209463749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:45:00.148 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c920d640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c4104520 0x7fe3c4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 -- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 msgr2=0x7fe3c419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 0x7fe3c419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 -- 172.21.15.43:0/4209463749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c4104520 msgr2=0x7fe3c4198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c4104520 0x7fe3c4198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c9a0e640 1 -- 172.21.15.43:0/4209463749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c419e850 con 0x7fe3c4103c00 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3c920d640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c4104520 0x7fe3c4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.144+0000 7fe3ca20f640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 0x7fe3c419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.145+0000 7fe3c9a0e640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4103c00 0x7fe3c41987b0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000e9f0 tx=0x7fe3c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.145+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000cd80 con 0x7fe3c4103c00 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.145+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c419eb40 con 0x7fe3c4103c00 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.145+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fe3c0010880 con 0x7fe3c4103c00 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.145+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0018760 con 0x7fe3c4103c00 2024-09-13T13:45:00.149 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.145+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a5200 con 0x7fe3c4103c00 2024-09-13T13:45:00.150 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.146+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c4103fe0 con 0x7fe3c4103c00 2024-09-13T13:45:00.150 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.147+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fe3c0010430 con 0x7fe3c4103c00 2024-09-13T13:45:00.150 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.147+0000 7fe3baffd640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fe3b007ae10 0x7fe3b007d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:00.150 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.147+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(737..737 src has 1..737) ==== 8575+0+0 (secure 0 0 0) 0x7fe3c009ed50 con 0x7fe3c4103c00 2024-09-13T13:45:00.150 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.147+0000 7fe3c920d640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fe3b007ae10 0x7fe3b007d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:00.151 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.148+0000 7fe3c920d640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fe3b007ae10 0x7fe3b007d2d0 secure :-1 s=READY pgs=3895 cs=0 l=1 rev1=1 crypto rx=0x7fe3c410e260 tx=0x7fe3b4009260 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:00.153 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.152+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fe3c0069b90 con 0x7fe3c4103c00 2024-09-13T13:45:00.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: osdmap e736: 8 total, 8 up, 8 in 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:45:00.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: osdmap e736: 8 total, 8 up, 8 in 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:44:59 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.341 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.340+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} v 0) -- 0x7fe3c4108020 con 0x7fe3c4103c00 2024-09-13T13:45:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:45:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:45:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: osdmap e736: 8 total, 8 up, 8 in 2024-09-13T13:45:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1043388708' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:45:00.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"} : dispatch 2024-09-13T13:45:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:45:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:44:59 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:00.947 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:00.946+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12}]=0 pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' created v738) ==== 176+0+0 (secure 0 0 0) 0x7fe3c006ea40 con 0x7fe3c4103c00 2024-09-13T13:45:01.061 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.060+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} v 0) -- 0x7fe3c4107bb0 con 0x7fe3c4103c00 2024-09-13T13:45:01.062 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.060+0000 7fe3baffd640 1 -- 172.21.15.43:0/4209463749 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12}]=0 pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' already exists v738) ==== 183+0+0 (secure 0 0 0) 0x7fe3c0010a30 con 0x7fe3c4103c00 2024-09-13T13:45:01.062 INFO:tasks.workunit.client.0.smithi043.stderr:pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' already exists 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fe3b007ae10 msgr2=0x7fe3b007d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fe3b007ae10 0x7fe3b007d2d0 secure :-1 s=READY pgs=3895 cs=0 l=1 rev1=1 crypto rx=0x7fe3c410e260 tx=0x7fe3b4009260 comp rx=0 tx=0).stop 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4103c00 msgr2=0x7fe3c41987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4103c00 0x7fe3c41987b0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000e9f0 tx=0x7fe3c000eeb0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 shutdown_connections 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fe3b007ae10 0x7fe3b007d2d0 unknown :-1 s=CLOSED pgs=3895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c410d050 0x7fe3c419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.064 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c4104520 0x7fe3c4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.065 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 --2- 172.21.15.43:0/4209463749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c4103c00 0x7fe3c41987b0 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.065 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 >> 172.21.15.43:0/4209463749 conn(0x7fe3c40fd610 msgr2=0x7fe3c40fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:01.065 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 shutdown_connections 2024-09-13T13:45:01.065 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.063+0000 7fe3cbc72640 1 -- 172.21.15.43:0/4209463749 wait complete. 2024-09-13T13:45:01.073 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 002fcfa4-dfdd-44b5-b965-f404930ab85a max_objects 10 2024-09-13T13:45:01.161 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 -- 172.21.15.43:0/2481299161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1838102cf0 msgr2=0x7f18381030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:01.161 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 --2- 172.21.15.43:0/2481299161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1838102cf0 0x7f18381030d0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f1828009900 tx=0x7f182802a410 comp rx=0 tx=0).stop 2024-09-13T13:45:01.161 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 -- 172.21.15.43:0/2481299161 shutdown_connections 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 --2- 172.21.15.43:0/2481299161 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1838109070 0x7f183810f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 --2- 172.21.15.43:0/2481299161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18381086f0 0x7f1838108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 --2- 172.21.15.43:0/2481299161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1838102cf0 0x7f18381030d0 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 -- 172.21.15.43:0/2481299161 >> 172.21.15.43:0/2481299161 conn(0x7f18380fe750 msgr2=0x7f1838100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 -- 172.21.15.43:0/2481299161 shutdown_connections 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.158+0000 7f183da47640 1 -- 172.21.15.43:0/2481299161 wait complete. 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 Processor -- start 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 -- start start 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1838102cf0 0x7f18380754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18381086f0 0x7f18380759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1838109070 0x7f1838075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:01.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1838112200 con 0x7f18381086f0 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1838112080 con 0x7f1838102cf0 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f183da47640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f1838112380 con 0x7f1838109070 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f182e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18381086f0 0x7f18380759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f18377fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1838109070 0x7f1838075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f182e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18381086f0 0x7f18380759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42266/0 (socket says 172.21.15.43:42266) 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f182e7fc640 1 -- 172.21.15.43:0/3742751888 learned_addr learned my addr 172.21.15.43:0/3742751888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.159+0000 7f1836ffd640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1838102cf0 0x7f18380754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f182e7fc640 1 -- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1838109070 msgr2=0x7f1838075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f182e7fc640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1838109070 0x7f1838075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f182e7fc640 1 -- 172.21.15.43:0/3742751888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1838102cf0 msgr2=0x7f18380754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f182e7fc640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1838102cf0 0x7f18380754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f182e7fc640 1 -- 172.21.15.43:0/3742751888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18381a6c30 con 0x7f18381086f0 2024-09-13T13:45:01.163 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f18377fe640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1838109070 0x7f1838075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:45:01.164 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f1836ffd640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1838102cf0 0x7f18380754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:45:01.164 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f182e7fc640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18381086f0 0x7f18380759e0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f182000c890 tx=0x7f182000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:01.164 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820004220 con 0x7f18381086f0 2024-09-13T13:45:01.164 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f1820007e00 con 0x7f18381086f0 2024-09-13T13:45:01.164 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18381a6f20 con 0x7f18381086f0 2024-09-13T13:45:01.164 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200047a0 con 0x7f18381086f0 2024-09-13T13:45:01.164 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.160+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18381a73e0 con 0x7f18381086f0 2024-09-13T13:45:01.165 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.161+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18381030d0 con 0x7f18381086f0 2024-09-13T13:45:01.165 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.162+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f18200079b0 con 0x7f18381086f0 2024-09-13T13:45:01.165 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.162+0000 7f183ca45640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f182407ae10 0x7f182407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:01.165 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.163+0000 7f1836ffd640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f182407ae10 0x7f182407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:01.165 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.163+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(738..738 src has 1..738) ==== 8950+0+0 (secure 0 0 0) 0x7f182009ea30 con 0x7f18381086f0 2024-09-13T13:45:01.165 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.163+0000 7f1836ffd640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f182407ae10 0x7f182407d2d0 secure :-1 s=READY pgs=3896 cs=0 l=1 rev1=1 crypto rx=0x7f1828000c00 tx=0x7f1828005a20 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:01.167 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.166+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f1820014030 con 0x7f18381086f0 2024-09-13T13:45:01.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[30106]: osdmap e737: 8 total, 8 up, 8 in 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} : dispatch 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[30106]: pgmap v1140: 164 pgs: 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[40255]: osdmap e737: 8 total, 8 up, 8 in 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:45:01.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} : dispatch 2024-09-13T13:45:01.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:00 smithi043 ceph-mon[40255]: pgmap v1140: 164 pgs: 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:45:01.354 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.353+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} v 0) -- 0x7f18380767a0 con 0x7f18381086f0 2024-09-13T13:45:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:00 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-76627-111"}]': finished 2024-09-13T13:45:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:00 smithi189 ceph-mon[32912]: osdmap e737: 8 total, 8 up, 8 in 2024-09-13T13:45:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:00 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:45:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:00 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:45:01.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:00 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:01.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:00 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} : dispatch 2024-09-13T13:45:01.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:00 smithi189 ceph-mon[32912]: pgmap v1140: 164 pgs: 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:45:01.949 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:01.947+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v739) ==== 223+0+0 (secure 0 0 0) 0x7f1820069730 con 0x7f18381086f0 2024-09-13T13:45:02.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.069+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} v 0) -- 0x7f183810ae50 con 0x7f18381086f0 2024-09-13T13:45:02.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12}]': finished 2024-09-13T13:45:02.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[30106]: osdmap e738: 8 total, 8 up, 8 in 2024-09-13T13:45:02.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} : dispatch 2024-09-13T13:45:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:45:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12}]': finished 2024-09-13T13:45:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[40255]: osdmap e738: 8 total, 8 up, 8 in 2024-09-13T13:45:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} : dispatch 2024-09-13T13:45:02.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:01 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:45:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12}]': finished 2024-09-13T13:45:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:01 smithi189 ceph-mon[32912]: osdmap e738: 8 total, 8 up, 8 in 2024-09-13T13:45:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/4209463749' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pg_num": 12} : dispatch 2024-09-13T13:45:02.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:01 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:45:02.963 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.961+0000 7f183ca45640 1 -- 172.21.15.43:0/3742751888 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v740) ==== 223+0+0 (secure 0 0 0) 0x7f182006e5e0 con 0x7f18381086f0 2024-09-13T13:45:02.963 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:45:02.967 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f182407ae10 msgr2=0x7f182407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:02.967 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f182407ae10 0x7f182407d2d0 secure :-1 s=READY pgs=3896 cs=0 l=1 rev1=1 crypto rx=0x7f1828000c00 tx=0x7f1828005a20 comp rx=0 tx=0).stop 2024-09-13T13:45:02.967 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18381086f0 msgr2=0x7f18380759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18381086f0 0x7f18380759e0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f182000c890 tx=0x7f182000cc90 comp rx=0 tx=0).stop 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 shutdown_connections 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f182407ae10 0x7f182407d2d0 unknown :-1 s=CLOSED pgs=3896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1838109070 0x7f1838075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18381086f0 0x7f18380759e0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 --2- 172.21.15.43:0/3742751888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1838102cf0 0x7f18380754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 >> 172.21.15.43:0/3742751888 conn(0x7f18380fe750 msgr2=0x7f18381001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 shutdown_connections 2024-09-13T13:45:02.968 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:02.966+0000 7f183da47640 1 -- 172.21.15.43:0/3742751888 wait complete. 2024-09-13T13:45:02.980 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable 002fcfa4-dfdd-44b5-b965-f404930ab85a rados 2024-09-13T13:45:03.073 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1956854687 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b68103ab0 msgr2=0x7f8b68103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:03.073 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1956854687 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b68103ab0 0x7f8b68103e90 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009900 tx=0x7f8b5802a410 comp rx=0 tx=0).stop 2024-09-13T13:45:03.073 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1956854687 shutdown_connections 2024-09-13T13:45:03.073 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1956854687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b6810d060 0x7f8b6810f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:03.073 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1956854687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b681044b0 0x7f8b6810ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:03.073 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1956854687 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b68103ab0 0x7f8b68103e90 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1956854687 >> 172.21.15.43:0/1956854687 conn(0x7f8b680fd660 msgr2=0x7f8b680ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1956854687 shutdown_connections 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.069+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1956854687 wait complete. 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 Processor -- start 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 -- start start 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b68103ab0 0x7f8b681987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b681044b0 0x7f8b68198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b6810d060 0x7f8b6819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68112190 con 0x7f8b681044b0 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68112010 con 0x7f8b68103ab0 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b6d3b4640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f8b68112310 con 0x7f8b6810d060 2024-09-13T13:45:03.074 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b681044b0 0x7f8b68198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b677fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b6810d060 0x7f8b6819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b681044b0 0x7f8b68198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42292/0 (socket says 172.21.15.43:42292) 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b677fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b6810d060 0x7f8b6819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:56088/0 (socket says 172.21.15.43:56088) 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.070+0000 7f8b5ffff640 1 -- 172.21.15.43:0/1987315878 learned_addr learned my addr 172.21.15.43:0/1987315878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b66ffd640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b68103ab0 0x7f8b681987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b5ffff640 1 -- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b6810d060 msgr2=0x7f8b6819e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b5ffff640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b6810d060 0x7f8b6819e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b5ffff640 1 -- 172.21.15.43:0/1987315878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b68103ab0 msgr2=0x7f8b681987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b5ffff640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b68103ab0 0x7f8b681987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b5ffff640 1 -- 172.21.15.43:0/1987315878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6819e840 con 0x7f8b681044b0 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b677fe640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b6810d060 0x7f8b6819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:45:03.075 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b5ffff640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b681044b0 0x7f8b68198ce0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000c970 tx=0x7f8b5000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:03.076 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50007bb0 con 0x7f8b681044b0 2024-09-13T13:45:03.076 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f8b50007d50 con 0x7f8b681044b0 2024-09-13T13:45:03.076 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6819eb30 con 0x7f8b681044b0 2024-09-13T13:45:03.076 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50004bd0 con 0x7f8b681044b0 2024-09-13T13:45:03.076 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.071+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681a51f0 con 0x7f8b681044b0 2024-09-13T13:45:03.076 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.072+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b68103e90 con 0x7f8b681044b0 2024-09-13T13:45:03.078 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.073+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f8b50004df0 con 0x7f8b681044b0 2024-09-13T13:45:03.078 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.073+0000 7f8b64ff9640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f8b5407ae10 0x7f8b5407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:03.078 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.073+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(740..740 src has 1..740) ==== 8950+0+0 (secure 0 0 0) 0x7f8b5009e700 con 0x7f8b681044b0 2024-09-13T13:45:03.078 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.073+0000 7f8b66ffd640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f8b5407ae10 0x7f8b5407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:03.078 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.074+0000 7f8b66ffd640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f8b5407ae10 0x7f8b5407d2d0 secure :-1 s=READY pgs=3897 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000c00 tx=0x7f8b580095f0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:03.079 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.078+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f8b50014030 con 0x7f8b681044b0 2024-09-13T13:45:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:45:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[30106]: osdmap e739: 8 total, 8 up, 8 in 2024-09-13T13:45:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:45:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[30106]: pgmap v1143: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:45:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[40255]: osdmap e739: 8 total, 8 up, 8 in 2024-09-13T13:45:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:45:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:02 smithi043 ceph-mon[40255]: pgmap v1143: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:03.267 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.266+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} v 0) -- 0x7f8b681998c0 con 0x7f8b681044b0 2024-09-13T13:45:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:45:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:02 smithi189 ceph-mon[32912]: osdmap e739: 8 total, 8 up, 8 in 2024-09-13T13:45:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:02 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:45:03.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:02 smithi189 ceph-mon[32912]: pgmap v1143: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:03.965 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:03.963+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]=0 enabled application 'rados' on pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' v741) ==== 213+0+0 (secure 0 0 0) 0x7f8b50069360 con 0x7f8b681044b0 2024-09-13T13:45:04.082 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.081+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} v 0) -- 0x7f8b68107c10 con 0x7f8b681044b0 2024-09-13T13:45:04.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:45:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:03 smithi043 ceph-mon[30106]: osdmap e740: 8 total, 8 up, 8 in 2024-09-13T13:45:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:03 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} : dispatch 2024-09-13T13:45:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:45:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:03 smithi043 ceph-mon[40255]: osdmap e740: 8 total, 8 up, 8 in 2024-09-13T13:45:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:03 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} : dispatch 2024-09-13T13:45:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3742751888' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:45:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:03 smithi189 ceph-mon[32912]: osdmap e740: 8 total, 8 up, 8 in 2024-09-13T13:45:04.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:03 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} : dispatch 2024-09-13T13:45:04.977 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.976+0000 7f8b64ff9640 1 -- 172.21.15.43:0/1987315878 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]=0 enabled application 'rados' on pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' v742) ==== 213+0+0 (secure 0 0 0) 0x7f8b5006e210 con 0x7f8b681044b0 2024-09-13T13:45:04.978 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.980+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f8b5407ae10 msgr2=0x7f8b5407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.980+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f8b5407ae10 0x7f8b5407d2d0 secure :-1 s=READY pgs=3897 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000c00 tx=0x7f8b580095f0 comp rx=0 tx=0).stop 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.980+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b681044b0 msgr2=0x7f8b68198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.980+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b681044b0 0x7f8b68198ce0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000c970 tx=0x7f8b5000cd70 comp rx=0 tx=0).stop 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 shutdown_connections 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f8b5407ae10 0x7f8b5407d2d0 unknown :-1 s=CLOSED pgs=3897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b6810d060 0x7f8b6819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b681044b0 0x7f8b68198ce0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 --2- 172.21.15.43:0/1987315878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b68103ab0 0x7f8b681987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 >> 172.21.15.43:0/1987315878 conn(0x7f8b680fd660 msgr2=0x7f8b681049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 shutdown_connections 2024-09-13T13:45:04.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:04.981+0000 7f8b6d3b4640 1 -- 172.21.15.43:0/1987315878 wait complete. 2024-09-13T13:45:04.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-09-13T13:45:05.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj1 /etc/passwd 2024-09-13T13:45:05.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj2 /etc/passwd 2024-09-13T13:45:05.091 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.092 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj3 /etc/passwd 2024-09-13T13:45:05.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj4 /etc/passwd 2024-09-13T13:45:05.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj5 /etc/passwd 2024-09-13T13:45:05.203 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.204 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj6 /etc/passwd 2024-09-13T13:45:05.241 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.241 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj7 /etc/passwd 2024-09-13T13:45:05.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]': finished 2024-09-13T13:45:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[30106]: osdmap e741: 8 total, 8 up, 8 in 2024-09-13T13:45:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} : dispatch 2024-09-13T13:45:05.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[30106]: pgmap v1146: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:45:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]': finished 2024-09-13T13:45:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[40255]: osdmap e741: 8 total, 8 up, 8 in 2024-09-13T13:45:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} : dispatch 2024-09-13T13:45:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:04 smithi043 ceph-mon[40255]: pgmap v1146: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:45:05.278 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.278 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj8 /etc/passwd 2024-09-13T13:45:05.317 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.317 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj9 /etc/passwd 2024-09-13T13:45:05.354 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:45:05.354 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put obj10 /etc/passwd 2024-09-13T13:45:05.391 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-13T13:45:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]': finished 2024-09-13T13:45:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:04 smithi189 ceph-mon[32912]: osdmap e741: 8 total, 8 up, 8 in 2024-09-13T13:45:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:04 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"} : dispatch 2024-09-13T13:45:05.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:04 smithi189 ceph-mon[32912]: pgmap v1146: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:45:05.979 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:45:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:45:06.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[30106]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:45:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]': finished 2024-09-13T13:45:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[30106]: osdmap e742: 8 total, 8 up, 8 in 2024-09-13T13:45:06.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[40255]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:45:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]': finished 2024-09-13T13:45:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[40255]: osdmap e742: 8 total, 8 up, 8 in 2024-09-13T13:45:06.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:06.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:45:05 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:45:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:05 smithi189 ceph-mon[32912]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T13:45:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:05 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1987315878' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "app": "rados"}]': finished 2024-09-13T13:45:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:05 smithi189 ceph-mon[32912]: osdmap e742: 8 total, 8 up, 8 in 2024-09-13T13:45:06.399 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:07.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:06 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:07.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:06 smithi043 ceph-mon[30106]: pgmap v1148: 176 pgs: 176 active+clean; 671 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 KiB/s wr, 2 op/s 2024-09-13T13:45:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:06 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:07.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:06 smithi043 ceph-mon[40255]: pgmap v1148: 176 pgs: 176 active+clean; 671 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 KiB/s wr, 2 op/s 2024-09-13T13:45:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:06 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:07.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:06 smithi189 ceph-mon[32912]: pgmap v1148: 176 pgs: 176 active+clean; 671 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 KiB/s wr, 2 op/s 2024-09-13T13:45:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:09 smithi189 ceph-mon[32912]: pgmap v1149: 176 pgs: 176 active+clean; 671 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-13T13:45:09.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:45:09 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=sqlstore.transactions t=2024-09-13T13:45:09.647078961Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-13T13:45:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:09 smithi043 ceph-mon[30106]: pgmap v1149: 176 pgs: 176 active+clean; 671 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-13T13:45:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:09 smithi043 ceph-mon[40255]: pgmap v1149: 176 pgs: 176 active+clean; 671 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-13T13:45:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:11 smithi189 ceph-mon[32912]: pgmap v1150: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-13T13:45:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:11 smithi043 ceph-mon[30106]: pgmap v1150: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-13T13:45:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:11 smithi043 ceph-mon[40255]: pgmap v1150: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-13T13:45:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:12 smithi189 ceph-mon[32912]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' is full (reached quota's max_objects: 10) 2024-09-13T13:45:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:12 smithi189 ceph-mon[32912]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:45:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:12 smithi189 ceph-mon[32912]: osdmap e743: 8 total, 8 up, 8 in 2024-09-13T13:45:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:12 smithi043 ceph-mon[30106]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' is full (reached quota's max_objects: 10) 2024-09-13T13:45:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:12 smithi043 ceph-mon[30106]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:45:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:12 smithi043 ceph-mon[30106]: osdmap e743: 8 total, 8 up, 8 in 2024-09-13T13:45:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:12 smithi043 ceph-mon[40255]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' is full (reached quota's max_objects: 10) 2024-09-13T13:45:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:12 smithi043 ceph-mon[40255]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:45:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:12 smithi043 ceph-mon[40255]: osdmap e743: 8 total, 8 up, 8 in 2024-09-13T13:45:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:13 smithi189 ceph-mon[32912]: pgmap v1152: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-13T13:45:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:13 smithi043 ceph-mon[30106]: pgmap v1152: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-13T13:45:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:13 smithi043 ceph-mon[40255]: pgmap v1152: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-13T13:45:15.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:15 smithi189 ceph-mon[32912]: pgmap v1153: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-13T13:45:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:15 smithi043 ceph-mon[30106]: pgmap v1153: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-13T13:45:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:15 smithi043 ceph-mon[40255]: pgmap v1153: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-13T13:45:15.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:45:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:45:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:45:16.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:45:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:45:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:17 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:17.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:17 smithi189 ceph-mon[32912]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-13T13:45:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:17 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:17 smithi043 ceph-mon[30106]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-13T13:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:17 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:17 smithi043 ceph-mon[40255]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-13T13:45:19.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:19 smithi189 ceph-mon[32912]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-13T13:45:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:19 smithi043 ceph-mon[30106]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-13T13:45:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:19 smithi043 ceph-mon[40255]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-13T13:45:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:21 smithi189 ceph-mon[32912]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-13T13:45:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:21 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:21.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:21 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:21 smithi043 ceph-mon[30106]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-13T13:45:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:21 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:21 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:21 smithi043 ceph-mon[40255]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-13T13:45:21.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:21 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:45:21.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:21 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:23.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:23 smithi189 ceph-mon[32912]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-09-13T13:45:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:23 smithi043 ceph-mon[30106]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-09-13T13:45:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:23 smithi043 ceph-mon[40255]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-09-13T13:45:25.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:25 smithi189 ceph-mon[32912]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-13T13:45:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:25 smithi043 ceph-mon[30106]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-13T13:45:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:25 smithi043 ceph-mon[40255]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-13T13:45:25.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:45:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:45:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:45:26.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:45:25 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:45:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:27 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:27 smithi189 ceph-mon[32912]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:27 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:27 smithi043 ceph-mon[30106]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:27 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:27 smithi043 ceph-mon[40255]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:29 smithi189 ceph-mon[32912]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:29 smithi043 ceph-mon[30106]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:29 smithi043 ceph-mon[40255]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:31 smithi189 ceph-mon[32912]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:31 smithi043 ceph-mon[30106]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:31 smithi043 ceph-mon[40255]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:33 smithi189 ceph-mon[32912]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:33 smithi043 ceph-mon[30106]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:33.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:33 smithi043 ceph-mon[40255]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:35.396 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=146494 2024-09-13T13:45:35.396 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 002fcfa4-dfdd-44b5-b965-f404930ab85a max_objects 100 2024-09-13T13:45:35.396 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put onemore /etc/passwd 2024-09-13T13:45:35.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 -- 172.21.15.43:0/92010289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 msgr2=0x7f6dd810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:35.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 --2- 172.21.15.43:0/92010289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 0x7f6dd810ca00 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009900 tx=0x7f6dcc02a410 comp rx=0 tx=0).stop 2024-09-13T13:45:35.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 -- 172.21.15.43:0/92010289 shutdown_connections 2024-09-13T13:45:35.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 --2- 172.21.15.43:0/92010289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd810d050 0x7f6dd810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:35.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 --2- 172.21.15.43:0/92010289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 0x7f6dd810ca00 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:35.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 --2- 172.21.15.43:0/92010289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6dd8103c00 0x7f6dd8103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:35.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 -- 172.21.15.43:0/92010289 >> 172.21.15.43:0/92010289 conn(0x7f6dd80fd630 msgr2=0x7f6dd80ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:35.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 -- 172.21.15.43:0/92010289 shutdown_connections 2024-09-13T13:45:35.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.490+0000 7f6dddc79640 1 -- 172.21.15.43:0/92010289 wait complete. 2024-09-13T13:45:35.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 Processor -- start 2024-09-13T13:45:35.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 -- start start 2024-09-13T13:45:35.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd8103c00 0x7f6dd8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:35.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 0x7f6dd8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:35.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6dd810d050 0x7f6dd819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:35.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd81121b0 con 0x7f6dd8103c00 2024-09-13T13:45:35.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd8112030 con 0x7f6dd810d050 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.491+0000 7f6dddc79640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd8112330 con 0x7f6dd8104520 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd8103c00 0x7f6dd8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd6ffd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 0x7f6dd8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd8103c00 0x7f6dd8198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44992/0 (socket says 172.21.15.43:44992) 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2812125122 learned_addr learned my addr 172.21.15.43:0/2812125122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd7fff640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6dd810d050 0x7f6dd819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 msgr2=0x7f6dd8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 0x7f6dd8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2812125122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6dd810d050 msgr2=0x7f6dd819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6dd810d050 0x7f6dd819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd819e7d0 con 0x7f6dd8103c00 2024-09-13T13:45:35.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd77fe640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd8103c00 0x7f6dd8198730 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c920 tx=0x7f6dc800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:35.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8007bb0 con 0x7f6dd8103c00 2024-09-13T13:45:35.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd819eac0 con 0x7f6dd8103c00 2024-09-13T13:45:35.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6dc8007d50 con 0x7f6dd8103c00 2024-09-13T13:45:35.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8018660 con 0x7f6dd8103c00 2024-09-13T13:45:35.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.492+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd81a5180 con 0x7f6dd8103c00 2024-09-13T13:45:35.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.493+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd8103fe0 con 0x7f6dd8103c00 2024-09-13T13:45:35.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.494+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f6dc800f450 con 0x7f6dd8103c00 2024-09-13T13:45:35.498 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.495+0000 7f6dd4ff9640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f6dc007ae10 0x7f6dc007d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:35.498 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.495+0000 7f6dd6ffd640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f6dc007ae10 0x7f6dc007d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:35.498 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.495+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(743..743 src has 1..743) ==== 8963+0+0 (secure 0 0 0) 0x7f6dc809eb50 con 0x7f6dd8103c00 2024-09-13T13:45:35.498 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.495+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=744}) -- 0x7f6dc0086700 con 0x7f6dd8103c00 2024-09-13T13:45:35.498 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.495+0000 7f6dd6ffd640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f6dc007ae10 0x7f6dc007d2d0 secure :-1 s=READY pgs=3909 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0057c0 tx=0x7f6dcc000bc0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:35.500 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.499+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f6dc8014030 con 0x7f6dd8103c00 2024-09-13T13:45:35.689 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:35.688+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} v 0) -- 0x7f6dd8108040 con 0x7f6dd8103c00 2024-09-13T13:45:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:35 smithi189 ceph-mon[32912]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:35 smithi043 ceph-mon[30106]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:35.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:45:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:45:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:45:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:35 smithi043 ceph-mon[40255]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:36.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:45:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:45:36.587 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:36.586+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v744) ==== 225+0+0 (secure 0 0 0) 0x7f6dc8069740 con 0x7f6dd8103c00 2024-09-13T13:45:36.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:36.587+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(744..744 src has 1..744) ==== 628+0+0 (secure 0 0 0) 0x7f6dc809c420 con 0x7f6dd8103c00 2024-09-13T13:45:36.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:36.587+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=745}) -- 0x7f6dc0087620 con 0x7f6dd8103c00 2024-09-13T13:45:36.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:36.703+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} v 0) -- 0x7f6dd8107ab0 con 0x7f6dd8103c00 2024-09-13T13:45:36.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:36 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} : dispatch 2024-09-13T13:45:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:36 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} : dispatch 2024-09-13T13:45:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:36 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} : dispatch 2024-09-13T13:45:37.041 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.040+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v745) ==== 225+0+0 (secure 0 0 0) 0x7f6dc806e5f0 con 0x7f6dd8103c00 2024-09-13T13:45:37.041 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 100 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:45:37.041 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.040+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 <== mon.0 v2:172.21.15.43:3300/0 10 ==== osd_map(745..745 src has 1..745) ==== 628+0+0 (secure 0 0 0) 0x7f6dc8060ed0 con 0x7f6dd8103c00 2024-09-13T13:45:37.041 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.040+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2812125122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=746}) -- 0x7f6dc0087830 con 0x7f6dd8103c00 2024-09-13T13:45:37.045 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f6dc007ae10 msgr2=0x7f6dc007d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:37.045 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f6dc007ae10 0x7f6dc007d2d0 secure :-1 s=READY pgs=3909 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0057c0 tx=0x7f6dcc000bc0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.045 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd8103c00 msgr2=0x7f6dd8198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:37.045 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd8103c00 0x7f6dd8198730 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c920 tx=0x7f6dc800cd20 comp rx=0 tx=0).stop 2024-09-13T13:45:37.045 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 shutdown_connections 2024-09-13T13:45:37.045 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6dd810d050 0x7f6dd819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.045 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f6dc007ae10 0x7f6dc007d2d0 unknown :-1 s=CLOSED pgs=3909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.046 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6dd8104520 0x7f6dd8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.046 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 --2- 172.21.15.43:0/2812125122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd8103c00 0x7f6dd8198730 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.046 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 >> 172.21.15.43:0/2812125122 conn(0x7f6dd80fd630 msgr2=0x7f6dd80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:37.047 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 shutdown_connections 2024-09-13T13:45:37.047 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.044+0000 7f6dddc79640 1 -- 172.21.15.43:0/2812125122 wait complete. 2024-09-13T13:45:37.057 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 146494 2024-09-13T13:45:37.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-09-13T13:45:37.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-09-13T13:45:37.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put twomore /etc/passwd 2024-09-13T13:45:37.096 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 002fcfa4-dfdd-44b5-b965-f404930ab85a max_bytes 100 2024-09-13T13:45:37.186 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 -- 172.21.15.43:0/2383388895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c101f90 msgr2=0x7fbf8c102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:37.186 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/2383388895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c101f90 0x7fbf8c102370 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009ca0 tx=0x7fbf7402a8c0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.187 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 -- 172.21.15.43:0/2383388895 shutdown_connections 2024-09-13T13:45:37.187 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/2383388895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf8c104d50 0x7fbf8c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.187 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/2383388895 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 0x7fbf8c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.187 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/2383388895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c101f90 0x7fbf8c102370 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.187 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 -- 172.21.15.43:0/2383388895 >> 172.21.15.43:0/2383388895 conn(0x7fbf8c0776a0 msgr2=0x7fbf8c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:37.188 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 -- 172.21.15.43:0/2383388895 shutdown_connections 2024-09-13T13:45:37.188 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.181+0000 7fbf92ee6640 1 -- 172.21.15.43:0/2383388895 wait complete. 2024-09-13T13:45:37.188 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 Processor -- start 2024-09-13T13:45:37.188 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 -- start start 2024-09-13T13:45:37.188 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf8c101f90 0x7fbf8c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 0x7fbf8c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c104d50 0x7fbf8c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c1110a0 con 0x7fbf8c104d50 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c110f20 con 0x7fbf8c101f90 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf92ee6640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c111220 con 0x7fbf8c1043b0 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf91483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c104d50 0x7fbf8c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 0x7fbf8c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 0x7fbf8c198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54272/0 (socket says 172.21.15.43:54272) 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf91483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c104d50 0x7fbf8c19e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37434/0 (socket says 172.21.15.43:37434) 2024-09-13T13:45:37.189 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.182+0000 7fbf8bfff640 1 -- 172.21.15.43:0/3585634004 learned_addr learned my addr 172.21.15.43:0/3585634004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf90c82640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf8c101f90 0x7fbf8c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf91483640 1 -- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 msgr2=0x7fbf8c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf91483640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 0x7fbf8c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf91483640 1 -- 172.21.15.43:0/3585634004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf8c101f90 msgr2=0x7fbf8c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf91483640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf8c101f90 0x7fbf8c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf91483640 1 -- 172.21.15.43:0/3585634004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c19e8a0 con 0x7fbf8c104d50 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf8bfff640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 0x7fbf8c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf91483640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c104d50 0x7fbf8c19e0d0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7fbf84007b20 tx=0x7fbf8400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8401b020 con 0x7fbf8c104d50 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fbf84004470 con 0x7fbf8c104d50 2024-09-13T13:45:37.190 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8c19eb90 con 0x7fbf8c104d50 2024-09-13T13:45:37.191 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84010ce0 con 0x7fbf8c104d50 2024-09-13T13:45:37.191 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.183+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8c0fed00 con 0x7fbf8c104d50 2024-09-13T13:45:37.191 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.184+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf8c102370 con 0x7fbf8c104d50 2024-09-13T13:45:37.191 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.185+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fbf8400c640 con 0x7fbf8c104d50 2024-09-13T13:45:37.192 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.186+0000 7fbf89ffb640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbf7807ae10 0x7fbf7807d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:45:37.192 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.186+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(745..745 src has 1..745) ==== 8963+0+0 (secure 0 0 0) 0x7fbf840a2070 con 0x7fbf8c104d50 2024-09-13T13:45:37.192 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.186+0000 7fbf90c82640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbf7807ae10 0x7fbf7807d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:45:37.192 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.186+0000 7fbf90c82640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbf7807ae10 0x7fbf7807d2d0 secure :-1 s=READY pgs=3911 cs=0 l=1 rev1=1 crypto rx=0x7fbf74006010 tx=0x7fbf74000f40 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:45:37.192 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.190+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fbf84025020 con 0x7fbf8c104d50 2024-09-13T13:45:37.379 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:37.378+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} v 0) -- 0x7fbf8c0fef60 con 0x7fbf8c104d50 2024-09-13T13:45:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]': finished 2024-09-13T13:45:37.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: osdmap e744: 8 total, 8 up, 8 in 2024-09-13T13:45:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} : dispatch 2024-09-13T13:45:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' no longer out of quota; removing NO_QUOTA flag 2024-09-13T13:45:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:45:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]': finished 2024-09-13T13:45:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: osdmap e745: 8 total, 8 up, 8 in 2024-09-13T13:45:37.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:37 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-13T13:45:37.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]': finished 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: osdmap e744: 8 total, 8 up, 8 in 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} : dispatch 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' no longer out of quota; removing NO_QUOTA flag 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]': finished 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: osdmap e745: 8 total, 8 up, 8 in 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-13T13:45:37.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]': finished 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: osdmap e744: 8 total, 8 up, 8 in 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"} : dispatch 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' no longer out of quota; removing NO_QUOTA flag 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2812125122' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "100"}]': finished 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: osdmap e745: 8 total, 8 up, 8 in 2024-09-13T13:45:37.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:37 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-13T13:45:38.042 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:38.041+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v746) ==== 221+0+0 (secure 0 0 0) 0x7fbf8406cd40 con 0x7fbf8c104d50 2024-09-13T13:45:38.172 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:38.171+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} v 0) -- 0x7fbf8c100330 con 0x7fbf8c104d50 2024-09-13T13:45:39.044 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.043+0000 7fbf89ffb640 1 -- 172.21.15.43:0/3585634004 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v747) ==== 221+0+0 (secure 0 0 0) 0x7fbf84071bf0 con 0x7fbf8c104d50 2024-09-13T13:45:39.044 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 100 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbf7807ae10 msgr2=0x7fbf7807d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbf7807ae10 0x7fbf7807d2d0 secure :-1 s=READY pgs=3911 cs=0 l=1 rev1=1 crypto rx=0x7fbf74006010 tx=0x7fbf74000f40 comp rx=0 tx=0).stop 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c104d50 msgr2=0x7fbf8c19e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c104d50 0x7fbf8c19e0d0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7fbf84007b20 tx=0x7fbf8400ee70 comp rx=0 tx=0).stop 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 shutdown_connections 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf8c104d50 0x7fbf8c19e0d0 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbf7807ae10 0x7fbf7807d2d0 unknown :-1 s=CLOSED pgs=3911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf8c1043b0 0x7fbf8c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 --2- 172.21.15.43:0/3585634004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf8c101f90 0x7fbf8c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:45:39.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.047+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 >> 172.21.15.43:0/3585634004 conn(0x7fbf8c0776a0 msgr2=0x7fbf8c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:45:39.050 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.048+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 shutdown_connections 2024-09-13T13:45:39.050 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:45:39.048+0000 7fbf92ee6640 1 -- 172.21.15.43:0/3585634004 wait complete. 2024-09-13T13:45:39.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-13T13:45:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]': finished 2024-09-13T13:45:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:39 smithi189 ceph-mon[32912]: osdmap e746: 8 total, 8 up, 8 in 2024-09-13T13:45:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:39 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-13T13:45:39.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:39 smithi189 ceph-mon[32912]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]': finished 2024-09-13T13:45:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[30106]: osdmap e746: 8 total, 8 up, 8 in 2024-09-13T13:45:39.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-13T13:45:39.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[30106]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]': finished 2024-09-13T13:45:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[40255]: osdmap e746: 8 total, 8 up, 8 in 2024-09-13T13:45:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-13T13:45:39.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:39 smithi043 ceph-mon[40255]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:40 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]': finished 2024-09-13T13:45:40.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:40 smithi189 ceph-mon[32912]: osdmap e747: 8 total, 8 up, 8 in 2024-09-13T13:45:40.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:40 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]': finished 2024-09-13T13:45:40.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:40 smithi043 ceph-mon[40255]: osdmap e747: 8 total, 8 up, 8 in 2024-09-13T13:45:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:40 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3585634004' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "100"}]': finished 2024-09-13T13:45:40.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:40 smithi043 ceph-mon[30106]: osdmap e747: 8 total, 8 up, 8 in 2024-09-13T13:45:41.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:41 smithi189 ceph-mon[32912]: pgmap v1170: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-13T13:45:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:41 smithi043 ceph-mon[30106]: pgmap v1170: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-13T13:45:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:41 smithi043 ceph-mon[40255]: pgmap v1170: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-13T13:45:42.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:42 smithi189 ceph-mon[32912]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' is full (reached quota's max_bytes: 100 B) 2024-09-13T13:45:42.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:42 smithi189 ceph-mon[32912]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:45:42.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:42 smithi189 ceph-mon[32912]: osdmap e748: 8 total, 8 up, 8 in 2024-09-13T13:45:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:42 smithi043 ceph-mon[30106]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' is full (reached quota's max_bytes: 100 B) 2024-09-13T13:45:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:42 smithi043 ceph-mon[30106]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:45:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:42 smithi043 ceph-mon[30106]: osdmap e748: 8 total, 8 up, 8 in 2024-09-13T13:45:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:42 smithi043 ceph-mon[40255]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' is full (reached quota's max_bytes: 100 B) 2024-09-13T13:45:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:42 smithi043 ceph-mon[40255]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:45:42.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:42 smithi043 ceph-mon[40255]: osdmap e748: 8 total, 8 up, 8 in 2024-09-13T13:45:43.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:43 smithi189 ceph-mon[32912]: pgmap v1172: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 941 B/s rd, 4.8 KiB/s wr, 1 op/s 2024-09-13T13:45:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:43 smithi043 ceph-mon[30106]: pgmap v1172: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 941 B/s rd, 4.8 KiB/s wr, 1 op/s 2024-09-13T13:45:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:43 smithi043 ceph-mon[40255]: pgmap v1172: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 941 B/s rd, 4.8 KiB/s wr, 1 op/s 2024-09-13T13:45:45.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:45 smithi189 ceph-mon[32912]: pgmap v1173: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 795 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-13T13:45:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:45 smithi043 ceph-mon[30106]: pgmap v1173: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 795 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-13T13:45:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:45 smithi043 ceph-mon[40255]: pgmap v1173: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 795 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-13T13:45:45.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:45:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:45:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:45:46.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:45:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:45:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:47 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:47.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:47 smithi189 ceph-mon[32912]: pgmap v1174: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-13T13:45:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:47 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:47.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:47 smithi043 ceph-mon[40255]: pgmap v1174: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-13T13:45:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:47 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:47 smithi043 ceph-mon[30106]: pgmap v1174: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-13T13:45:49.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:49 smithi189 ceph-mon[32912]: pgmap v1175: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-13T13:45:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:49 smithi043 ceph-mon[30106]: pgmap v1175: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-13T13:45:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:49 smithi043 ceph-mon[40255]: pgmap v1175: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-09-13T13:45:51.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:51 smithi189 ceph-mon[32912]: pgmap v1176: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:45:51.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:51 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:51 smithi043 ceph-mon[30106]: pgmap v1176: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:45:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:51 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:51 smithi043 ceph-mon[40255]: pgmap v1176: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:45:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:51 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:45:53.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:53 smithi189 ceph-mon[32912]: pgmap v1177: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 980 B/s rd, 0 op/s 2024-09-13T13:45:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:53 smithi043 ceph-mon[30106]: pgmap v1177: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 980 B/s rd, 0 op/s 2024-09-13T13:45:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:53 smithi043 ceph-mon[40255]: pgmap v1177: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 980 B/s rd, 0 op/s 2024-09-13T13:45:55.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:55 smithi189 ceph-mon[32912]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:45:55.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:55 smithi043 ceph-mon[30106]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:45:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:55 smithi043 ceph-mon[40255]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:45:55.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:45:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:45:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:45:56.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:45:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:45:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:57 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:57.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:57 smithi189 ceph-mon[32912]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:57 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:57 smithi043 ceph-mon[40255]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:57 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:45:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:57 smithi043 ceph-mon[30106]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:45:59.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:45:59 smithi189 ceph-mon[32912]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:45:59 smithi043 ceph-mon[40255]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:45:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:45:59 smithi043 ceph-mon[30106]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:46:00.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:00 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:46:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:00 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:46:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:00 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:46:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:01 smithi189 ceph-mon[32912]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:01 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:46:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:01 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:46:01.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:01 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:46:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[30106]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:46:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:46:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:46:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[40255]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:46:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:46:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:01 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:46:03.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:03 smithi189 ceph-mon[32912]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:46:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:03 smithi043 ceph-mon[30106]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:46:03.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:03 smithi043 ceph-mon[40255]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:46:05.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:05 smithi189 ceph-mon[32912]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:05.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:05 smithi043 ceph-mon[30106]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:05 smithi043 ceph-mon[40255]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:05.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:46:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:46:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:46:06.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:46:06 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:46:06.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:06 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:06 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:06 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:07 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:07.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:07 smithi189 ceph-mon[32912]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:07 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:07 smithi043 ceph-mon[30106]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:07 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:07 smithi043 ceph-mon[40255]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:09.063 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=146673 2024-09-13T13:46:09.063 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 002fcfa4-dfdd-44b5-b965-f404930ab85a max_bytes 0 2024-09-13T13:46:09.063 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put two /etc/passwd 2024-09-13T13:46:09.166 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.161+0000 7ff19555d640 1 -- 172.21.15.43:0/841457300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff190104520 msgr2=0x7ff19010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:09.167 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.161+0000 7ff19555d640 1 --2- 172.21.15.43:0/841457300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff190104520 0x7ff19010ca00 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7ff178009900 tx=0x7ff17802a3f0 comp rx=0 tx=0).stop 2024-09-13T13:46:09.167 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.161+0000 7ff19555d640 1 -- 172.21.15.43:0/841457300 shutdown_connections 2024-09-13T13:46:09.167 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.161+0000 7ff19555d640 1 --2- 172.21.15.43:0/841457300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff19010d050 0x7ff19010f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:09.167 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.161+0000 7ff19555d640 1 --2- 172.21.15.43:0/841457300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff190104520 0x7ff19010ca00 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:09.167 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.161+0000 7ff19555d640 1 --2- 172.21.15.43:0/841457300 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 0x7ff190103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:09.168 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.161+0000 7ff19555d640 1 -- 172.21.15.43:0/841457300 >> 172.21.15.43:0/841457300 conn(0x7ff1900fd630 msgr2=0x7ff1900ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:09.168 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.162+0000 7ff19555d640 1 -- 172.21.15.43:0/841457300 shutdown_connections 2024-09-13T13:46:09.168 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.162+0000 7ff19555d640 1 -- 172.21.15.43:0/841457300 wait complete. 2024-09-13T13:46:09.168 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.162+0000 7ff19555d640 1 Processor -- start 2024-09-13T13:46:09.168 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.162+0000 7ff19555d640 1 -- start start 2024-09-13T13:46:09.168 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff19555d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 0x7ff1901018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:09.168 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff19555d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff190104520 0x7ff190101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff19555d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19010d050 0x7ff190102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff19555d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff190112200 con 0x7ff19010d050 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff19555d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff190112080 con 0x7ff190104520 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff19555d640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7ff190112380 con 0x7ff190103c00 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18effd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 0x7ff1901018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19010d050 0x7ff190102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19010d050 0x7ff190102320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56630/0 (socket says 172.21.15.43:56630) 2024-09-13T13:46:09.169 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18effd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 0x7ff1901018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:40408/0 (socket says 172.21.15.43:40408) 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 -- 172.21.15.43:0/841668073 learned_addr learned my addr 172.21.15.43:0/841668073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18e7fc640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff190104520 0x7ff190101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 -- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 msgr2=0x7ff1901018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 0x7ff1901018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 -- 172.21.15.43:0/841668073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff190104520 msgr2=0x7ff190101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff190104520 0x7ff190101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1901a6c30 con 0x7ff19010d050 2024-09-13T13:46:09.170 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18e7fc640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff190104520 0x7ff190101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18effd640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 0x7ff1901018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff18f7fe640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19010d050 0x7ff190102320 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7ff18800eaa0 tx=0x7ff18800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18800cdb0 con 0x7ff19010d050 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ff188005020 con 0x7ff19010d050 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.163+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1901a6f20 con 0x7ff19010d050 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.164+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880116d0 con 0x7ff19010d050 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.164+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1901a7390 con 0x7ff19010d050 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.165+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff190103fe0 con 0x7ff19010d050 2024-09-13T13:46:09.171 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.165+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7ff188004a40 con 0x7ff19010d050 2024-09-13T13:46:09.173 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.166+0000 7ff187fff640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff17c07ae10 0x7ff17c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:09.173 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.166+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(748..748 src has 1..748) ==== 8963+0+0 (secure 0 0 0) 0x7ff18809ebd0 con 0x7ff19010d050 2024-09-13T13:46:09.173 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.166+0000 7ff18effd640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff17c07ae10 0x7ff17c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:09.173 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.166+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=749}) -- 0x7ff17c086700 con 0x7ff19010d050 2024-09-13T13:46:09.173 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.167+0000 7ff18effd640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff17c07ae10 0x7ff17c07d2d0 secure :-1 s=READY pgs=3913 cs=0 l=1 rev1=1 crypto rx=0x7ff1800095f0 tx=0x7ff180005f90 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:09.173 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.171+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff188016030 con 0x7ff19010d050 2024-09-13T13:46:09.360 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.359+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} v 0) -- 0x7ff190102ba0 con 0x7ff19010d050 2024-09-13T13:46:09.585 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.584+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v749) ==== 217+0+0 (secure 0 0 0) 0x7ff188005a10 con 0x7ff19010d050 2024-09-13T13:46:09.585 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.584+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(749..749 src has 1..749) ==== 628+0+0 (secure 0 0 0) 0x7ff1880880c0 con 0x7ff19010d050 2024-09-13T13:46:09.586 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.584+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=750}) -- 0x7ff17c087620 con 0x7ff19010d050 2024-09-13T13:46:09.702 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:09.701+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} v 0) -- 0x7ff190108040 con 0x7ff19010d050 2024-09-13T13:46:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:09 smithi189 ceph-mon[32912]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:46:09.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:09 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:46:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:09 smithi043 ceph-mon[40255]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:46:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:09 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:46:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:09 smithi043 ceph-mon[30106]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-13T13:46:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:09 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:46:10.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.589+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v750) ==== 217+0+0 (secure 0 0 0) 0x7ff1880697c0 con 0x7ff19010d050 2024-09-13T13:46:10.591 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:46:10.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.590+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 <== mon.0 v2:172.21.15.43:3300/0 10 ==== osd_map(750..750 src has 1..750) ==== 628+0+0 (secure 0 0 0) 0x7ff18809c4a0 con 0x7ff19010d050 2024-09-13T13:46:10.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.590+0000 7ff187fff640 1 -- 172.21.15.43:0/841668073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=751}) -- 0x7ff17c087830 con 0x7ff19010d050 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.594+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff17c07ae10 msgr2=0x7ff17c07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.594+0000 7ff19555d640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff17c07ae10 0x7ff17c07d2d0 secure :-1 s=READY pgs=3913 cs=0 l=1 rev1=1 crypto rx=0x7ff1800095f0 tx=0x7ff180005f90 comp rx=0 tx=0).stop 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.594+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19010d050 msgr2=0x7ff190102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.594+0000 7ff19555d640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19010d050 0x7ff190102320 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7ff18800eaa0 tx=0x7ff18800ef60 comp rx=0 tx=0).stop 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.595+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 shutdown_connections 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.595+0000 7ff19555d640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19010d050 0x7ff190102320 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.595+0000 7ff19555d640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff190104520 0x7ff190101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.595+0000 7ff19555d640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff17c07ae10 0x7ff17c07d2d0 unknown :-1 s=CLOSED pgs=3913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.595+0000 7ff19555d640 1 --2- 172.21.15.43:0/841668073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff190103c00 0x7ff1901018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.595+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 >> 172.21.15.43:0/841668073 conn(0x7ff1900fd630 msgr2=0x7ff1900fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:10.597 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.595+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 shutdown_connections 2024-09-13T13:46:10.597 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.596+0000 7ff19555d640 1 -- 172.21.15.43:0/841668073 wait complete. 2024-09-13T13:46:10.612 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 002fcfa4-dfdd-44b5-b965-f404930ab85a max_objects 0 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 -- 172.21.15.43:0/818220159 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0104d50 msgr2=0x7fbde010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 --2- 172.21.15.43:0/818220159 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0104d50 0x7fbde010e750 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009860 tx=0x7fbdd802a340 comp rx=0 tx=0).stop 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 -- 172.21.15.43:0/818220159 shutdown_connections 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 --2- 172.21.15.43:0/818220159 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0104d50 0x7fbde010e750 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 --2- 172.21.15.43:0/818220159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbde01043b0 0x7fbde0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 --2- 172.21.15.43:0/818220159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde0101f90 0x7fbde0102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 -- 172.21.15.43:0/818220159 >> 172.21.15.43:0/818220159 conn(0x7fbde00776a0 msgr2=0x7fbde0077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:10.703 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 -- 172.21.15.43:0/818220159 shutdown_connections 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.700+0000 7fbde7518640 1 -- 172.21.15.43:0/818220159 wait complete. 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 Processor -- start 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 -- start start 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0101f90 0x7fbde0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde01043b0 0x7fbde0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbde0104d50 0x7fbde019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbde01110e0 con 0x7fbde01043b0 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0110f60 con 0x7fbde0104d50 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde7518640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fbde0111260 con 0x7fbde0101f90 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0101f90 0x7fbde0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0101f90 0x7fbde0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:40420/0 (socket says 172.21.15.43:40420) 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde4ab3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde01043b0 0x7fbde0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:10.704 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 -- 172.21.15.43:0/2768518327 learned_addr learned my addr 172.21.15.43:0/2768518327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 -- 172.21.15.43:0/2768518327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbde0104d50 msgr2=0x7fbde019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbde0104d50 0x7fbde019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 -- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde01043b0 msgr2=0x7fbde0198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde01043b0 0x7fbde0198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde019e800 con 0x7fbde0101f90 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde4ab3640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde01043b0 0x7fbde0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.701+0000 7fbde52b4640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0101f90 0x7fbde0198760 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e9c0 tx=0x7fbdd000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.702+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000cde0 con 0x7fbde0101f90 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.702+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fbdd0004470 con 0x7fbde0101f90 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.702+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde019eaf0 con 0x7fbde0101f90 2024-09-13T13:46:10.705 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.702+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0003e50 con 0x7fbde0101f90 2024-09-13T13:46:10.706 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.702+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a5390 con 0x7fbde0101f90 2024-09-13T13:46:10.707 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.703+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde0102370 con 0x7fbde0101f90 2024-09-13T13:46:10.707 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.704+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fbdd0004000 con 0x7fbde0101f90 2024-09-13T13:46:10.707 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.704+0000 7fbdde7fc640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbdc807afb0 0x7fbdc807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:10.707 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.704+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(750..750 src has 1..750) ==== 8963+0+0 (secure 0 0 0) 0x7fbdd009ed60 con 0x7fbde0101f90 2024-09-13T13:46:10.707 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.704+0000 7fbde4ab3640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbdc807afb0 0x7fbdc807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:10.707 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.704+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=751}) -- 0x7fbdc8086920 con 0x7fbde0101f90 2024-09-13T13:46:10.707 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.705+0000 7fbde4ab3640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbdc807afb0 0x7fbdc807d470 secure :-1 s=READY pgs=3914 cs=0 l=1 rev1=1 crypto rx=0x7fbde0103c80 tx=0x7fbdd400a590 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:10.710 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.709+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fbdd0069a30 con 0x7fbde0101f90 2024-09-13T13:46:10.897 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:10.896+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} v 0) -- 0x7fbde010ad40 con 0x7fbde0101f90 2024-09-13T13:46:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:46:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:10 smithi189 ceph-mon[32912]: osdmap e749: 8 total, 8 up, 8 in 2024-09-13T13:46:10.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:10 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:46:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:46:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:10 smithi043 ceph-mon[40255]: osdmap e749: 8 total, 8 up, 8 in 2024-09-13T13:46:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:10 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:46:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:46:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:10 smithi043 ceph-mon[30106]: osdmap e749: 8 total, 8 up, 8 in 2024-09-13T13:46:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:10 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:46:11.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:11.592+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(751..751 src has 1..751) ==== 628+0+0 (secure 0 0 0) 0x7fbdd00035f0 con 0x7fbde0101f90 2024-09-13T13:46:11.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:11.592+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=752}) -- 0x7fbdc8087490 con 0x7fbde0101f90 2024-09-13T13:46:11.599 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:11.598+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v751) ==== 221+0+0 (secure 0 0 0) 0x7fbdd006e8e0 con 0x7fbde0101f90 2024-09-13T13:46:11.713 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:11.712+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} v 0) -- 0x7fbde010a7b0 con 0x7fbde0101f90 2024-09-13T13:46:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:11 smithi189 ceph-mon[32912]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:46:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:46:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:11 smithi189 ceph-mon[32912]: osdmap e750: 8 total, 8 up, 8 in 2024-09-13T13:46:11.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:11 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2768518327' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:11.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:11 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[30106]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:46:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:46:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[30106]: osdmap e750: 8 total, 8 up, 8 in 2024-09-13T13:46:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2768518327' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[40255]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:46:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/841668073' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:46:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[40255]: osdmap e750: 8 total, 8 up, 8 in 2024-09-13T13:46:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2768518327' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:11 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:12.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.048+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 9 ==== osd_map(752..752 src has 1..752) ==== 628+0+0 (secure 0 0 0) 0x7fbdd009c660 con 0x7fbde0101f90 2024-09-13T13:46:12.049 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.048+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=753}) -- 0x7fbdc8087a90 con 0x7fbde0101f90 2024-09-13T13:46:12.053 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.052+0000 7fbdde7fc640 1 -- 172.21.15.43:0/2768518327 <== mon.2 v2:172.21.15.43:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v752) ==== 221+0+0 (secure 0 0 0) 0x7fbdd008c170 con 0x7fbde0101f90 2024-09-13T13:46:12.053 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:46:12.057 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbdc807afb0 msgr2=0x7fbdc807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:12.057 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbdc807afb0 0x7fbdc807d470 secure :-1 s=READY pgs=3914 cs=0 l=1 rev1=1 crypto rx=0x7fbde0103c80 tx=0x7fbdd400a590 comp rx=0 tx=0).stop 2024-09-13T13:46:12.057 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0101f90 msgr2=0x7fbde0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:12.057 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0101f90 0x7fbde0198760 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e9c0 tx=0x7fbdd000ee80 comp rx=0 tx=0).stop 2024-09-13T13:46:12.057 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 shutdown_connections 2024-09-13T13:46:12.058 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbde0104d50 0x7fbde019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.058 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde01043b0 0x7fbde0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.058 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fbdc807afb0 0x7fbdc807d470 unknown :-1 s=CLOSED pgs=3914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.058 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 --2- 172.21.15.43:0/2768518327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbde0101f90 0x7fbde0198760 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.058 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 >> 172.21.15.43:0/2768518327 conn(0x7fbde00776a0 msgr2=0x7fbde006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:12.058 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 shutdown_connections 2024-09-13T13:46:12.058 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.056+0000 7fbde7518640 1 -- 172.21.15.43:0/2768518327 wait complete. 2024-09-13T13:46:12.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 146673 2024-09-13T13:46:12.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-09-13T13:46:12.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-09-13T13:46:12.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put three /etc/passwd 2024-09-13T13:46:12.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-09-13T13:46:12.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ pp=8ef39e5e-57a7-4748-b29c-009db0c54857 2024-09-13T13:46:12.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create 8ef39e5e-57a7-4748-b29c-009db0c54857 12 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 -- 172.21.15.43:0/924174438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838102cf0 msgr2=0x7fb8381030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/924174438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838102cf0 0x7fb8381030d0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fb828009900 tx=0x7fb82802a3f0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 -- 172.21.15.43:0/924174438 shutdown_connections 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/924174438 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838109070 0x7fb83810f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/924174438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8381086f0 0x7fb838108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/924174438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838102cf0 0x7fb8381030d0 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 -- 172.21.15.43:0/924174438 >> 172.21.15.43:0/924174438 conn(0x7fb8380fe750 msgr2=0x7fb838100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 -- 172.21.15.43:0/924174438 shutdown_connections 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.194+0000 7fb83e4b1640 1 -- 172.21.15.43:0/924174438 wait complete. 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 Processor -- start 2024-09-13T13:46:12.196 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 -- start start 2024-09-13T13:46:12.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838102cf0 0x7fb8380754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:12.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8381086f0 0x7fb8380759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:12.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838109070 0x7fb838075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:12.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb838112200 con 0x7fb838109070 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb838112080 con 0x7fb8381086f0 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83e4b1640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fb838112380 con 0x7fb838102cf0 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb837fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838102cf0 0x7fb8380754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83ca4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838109070 0x7fb838075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb837fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838102cf0 0x7fb8380754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:40478/0 (socket says 172.21.15.43:40478) 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb83ca4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838109070 0x7fb838075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56658/0 (socket says 172.21.15.43:56658) 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb837fff640 1 -- 172.21.15.43:0/3554781073 learned_addr learned my addr 172.21.15.43:0/3554781073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.195+0000 7fb8377fe640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8381086f0 0x7fb8380759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:12.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83ca4e640 1 -- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838102cf0 msgr2=0x7fb8380754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83ca4e640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838102cf0 0x7fb8380754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83ca4e640 1 -- 172.21.15.43:0/3554781073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8381086f0 msgr2=0x7fb8380759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83ca4e640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8381086f0 0x7fb8380759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83ca4e640 1 -- 172.21.15.43:0/3554781073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8381a6b10 con 0x7fb838109070 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb8377fe640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8381086f0 0x7fb8380759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb837fff640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838102cf0 0x7fb8380754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83ca4e640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838109070 0x7fb838075f20 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fb83000ec50 tx=0x7fb83000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830003b70 con 0x7fb838109070 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb830003d10 con 0x7fb838109070 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8381a6e00 con 0x7fb838109070 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830010690 con 0x7fb838109070 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.196+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381a7270 con 0x7fb838109070 2024-09-13T13:46:12.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.197+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8381030d0 con 0x7fb838109070 2024-09-13T13:46:12.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.198+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fb830003fc0 con 0x7fb838109070 2024-09-13T13:46:12.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.198+0000 7fb8357fa640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb82007ae10 0x7fb82007d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:12.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.198+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(752..752 src has 1..752) ==== 8963+0+0 (secure 0 0 0) 0x7fb83009f000 con 0x7fb838109070 2024-09-13T13:46:12.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.198+0000 7fb837fff640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb82007ae10 0x7fb82007d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:12.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.199+0000 7fb837fff640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb82007ae10 0x7fb82007d2d0 secure :-1 s=READY pgs=3916 cs=0 l=1 rev1=1 crypto rx=0x7fb828000c00 tx=0x7fb8280058d0 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:12.204 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.203+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb830069c50 con 0x7fb838109070 2024-09-13T13:46:12.390 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:12.389+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} v 0) -- 0x7fb83802e5c0 con 0x7fb838109070 2024-09-13T13:46:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:46:12.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: osdmap e751: 8 total, 8 up, 8 in 2024-09-13T13:46:12.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2768518327' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:12.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:12.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' no longer out of quota; removing NO_QUOTA flag 2024-09-13T13:46:12.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:46:12.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:46:12.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: osdmap e752: 8 total, 8 up, 8 in 2024-09-13T13:46:12.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:12 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} : dispatch 2024-09-13T13:46:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: osdmap e751: 8 total, 8 up, 8 in 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2768518327' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' no longer out of quota; removing NO_QUOTA flag 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: osdmap e752: 8 total, 8 up, 8 in 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} : dispatch 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: osdmap e751: 8 total, 8 up, 8 in 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2768518327' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:46:12.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' no longer out of quota; removing NO_QUOTA flag 2024-09-13T13:46:12.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:46:12.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:46:12.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: osdmap e752: 8 total, 8 up, 8 in 2024-09-13T13:46:12.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:12 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} : dispatch 2024-09-13T13:46:13.053 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.052+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12}]=0 pool '8ef39e5e-57a7-4748-b29c-009db0c54857' created v753) ==== 176+0+0 (secure 0 0 0) 0x7fb83006eb00 con 0x7fb838109070 2024-09-13T13:46:13.174 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.172+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} v 0) -- 0x7fb83802e790 con 0x7fb838109070 2024-09-13T13:46:13.174 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.173+0000 7fb8357fa640 1 -- 172.21.15.43:0/3554781073 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12}]=0 pool '8ef39e5e-57a7-4748-b29c-009db0c54857' already exists v753) ==== 183+0+0 (secure 0 0 0) 0x7fb83009c900 con 0x7fb838109070 2024-09-13T13:46:13.174 INFO:tasks.workunit.client.0.smithi043.stderr:pool '8ef39e5e-57a7-4748-b29c-009db0c54857' already exists 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb82007ae10 msgr2=0x7fb82007d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb82007ae10 0x7fb82007d2d0 secure :-1 s=READY pgs=3916 cs=0 l=1 rev1=1 crypto rx=0x7fb828000c00 tx=0x7fb8280058d0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838109070 msgr2=0x7fb838075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838109070 0x7fb838075f20 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fb83000ec50 tx=0x7fb83000c6a0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 shutdown_connections 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb838109070 0x7fb838075f20 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8381086f0 0x7fb8380759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fb82007ae10 0x7fb82007d2d0 unknown :-1 s=CLOSED pgs=3916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 --2- 172.21.15.43:0/3554781073 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb838102cf0 0x7fb8380754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 >> 172.21.15.43:0/3554781073 conn(0x7fb8380fe750 msgr2=0x7fb838100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:13.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 shutdown_connections 2024-09-13T13:46:13.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.175+0000 7fb83e4b1640 1 -- 172.21.15.43:0/3554781073 wait complete. 2024-09-13T13:46:13.186 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable 8ef39e5e-57a7-4748-b29c-009db0c54857 rados 2024-09-13T13:46:13.272 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 -- 172.21.15.43:0/2008125345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140107f60 msgr2=0x7f514010e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:13.272 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 --2- 172.21.15.43:0/2008125345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140107f60 0x7f514010e760 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f5138009ca0 tx=0x7f513802a8c0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.272 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 -- 172.21.15.43:0/2008125345 shutdown_connections 2024-09-13T13:46:13.272 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 --2- 172.21.15.43:0/2008125345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140107f60 0x7f514010e760 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.272 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 --2- 172.21.15.43:0/2008125345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51401075e0 0x7f5140107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.272 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 --2- 172.21.15.43:0/2008125345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5140101be0 0x7f5140101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.273 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 -- 172.21.15.43:0/2008125345 >> 172.21.15.43:0/2008125345 conn(0x7f51400fd640 msgr2=0x7f51400ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:13.273 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 -- 172.21.15.43:0/2008125345 shutdown_connections 2024-09-13T13:46:13.273 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 -- 172.21.15.43:0/2008125345 wait complete. 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.271+0000 7f5144ccd640 1 Processor -- start 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5144ccd640 1 -- start start 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5144ccd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140101be0 0x7f51401994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5144ccd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51401075e0 0x7f5140199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5144ccd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5140107f60 0x7f51401a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5144ccd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51401110f0 con 0x7f5140107f60 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5144ccd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5140110f70 con 0x7f51401075e0 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5144ccd640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f5140111270 con 0x7f5140101be0 2024-09-13T13:46:13.275 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140101be0 0x7f51401994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5140107f60 0x7f51401a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140101be0 0x7f51401994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:40510/0 (socket says 172.21.15.43:40510) 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 -- 172.21.15.43:0/59055804 learned_addr learned my addr 172.21.15.43:0/59055804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5135d9b640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51401075e0 0x7f5140199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 -- 172.21.15.43:0/59055804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51401075e0 msgr2=0x7f5140199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51401075e0 0x7f5140199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 -- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5140107f60 msgr2=0x7f51401a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5140107f60 0x7f51401a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513e59c640 1 -- 172.21.15.43:0/59055804 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51401a4aa0 con 0x7f5140101be0 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f5135d9b640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51401075e0 0x7f5140199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.272+0000 7f513ed9d640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5140107f60 0x7f51401a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.273+0000 7f513e59c640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140101be0 0x7f51401994e0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f513000d770 tx=0x7f513000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.273+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130017070 con 0x7f5140101be0 2024-09-13T13:46:13.276 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.273+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f5130005470 con 0x7f5140101be0 2024-09-13T13:46:13.277 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.273+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51401a4c70 con 0x7f5140101be0 2024-09-13T13:46:13.277 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.273+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130004b40 con 0x7f5140101be0 2024-09-13T13:46:13.277 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.273+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f51401a50e0 con 0x7f5140101be0 2024-09-13T13:46:13.277 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.274+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5140101fc0 con 0x7f5140101be0 2024-09-13T13:46:13.278 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.274+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f5130004ce0 con 0x7f5140101be0 2024-09-13T13:46:13.278 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.275+0000 7f5137fff640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f512c07ae10 0x7f512c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:13.278 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.275+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(753..753 src has 1..753) ==== 9338+0+0 (secure 0 0 0) 0x7f513009e650 con 0x7f5140101be0 2024-09-13T13:46:13.278 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.275+0000 7f5135d9b640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f512c07ae10 0x7f512c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:13.278 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.275+0000 7f5135d9b640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f512c07ae10 0x7f512c07d2d0 secure :-1 s=READY pgs=3917 cs=0 l=1 rev1=1 crypto rx=0x7f514006bec0 tx=0x7f5128005f90 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:13.280 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.279+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f5130069130 con 0x7f5140101be0 2024-09-13T13:46:13.465 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:13.464+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} v 0) -- 0x7f514006b6d0 con 0x7f5140101be0 2024-09-13T13:46:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:13 smithi189 ceph-mon[32912]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12}]': finished 2024-09-13T13:46:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:13 smithi189 ceph-mon[32912]: osdmap e753: 8 total, 8 up, 8 in 2024-09-13T13:46:13.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} : dispatch 2024-09-13T13:46:13.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:13 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/59055804' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:13.899 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:13 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[30106]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12}]': finished 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[30106]: osdmap e753: 8 total, 8 up, 8 in 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} : dispatch 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/59055804' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[40255]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12}]': finished 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[40255]: osdmap e753: 8 total, 8 up, 8 in 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3554781073' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pg_num": 12} : dispatch 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/59055804' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:13 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:14.060 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:14.059+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]=0 enabled application 'rados' on pool '8ef39e5e-57a7-4748-b29c-009db0c54857' v754) ==== 213+0+0 (secure 0 0 0) 0x7f513006dfe0 con 0x7f5140101be0 2024-09-13T13:46:14.181 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:14.180+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} v 0) -- 0x7f514002e750 con 0x7f5140101be0 2024-09-13T13:46:15.066 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.065+0000 7f5137fff640 1 -- 172.21.15.43:0/59055804 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]=0 enabled application 'rados' on pool '8ef39e5e-57a7-4748-b29c-009db0c54857' v755) ==== 213+0+0 (secure 0 0 0) 0x7f513009bde0 con 0x7f5140101be0 2024-09-13T13:46:15.066 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool '8ef39e5e-57a7-4748-b29c-009db0c54857' 2024-09-13T13:46:15.069 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f512c07ae10 msgr2=0x7f512c07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f512c07ae10 0x7f512c07d2d0 secure :-1 s=READY pgs=3917 cs=0 l=1 rev1=1 crypto rx=0x7f514006bec0 tx=0x7f5128005f90 comp rx=0 tx=0).stop 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140101be0 msgr2=0x7f51401994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140101be0 0x7f51401994e0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f513000d770 tx=0x7f513000dc30 comp rx=0 tx=0).stop 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 shutdown_connections 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5140107f60 0x7f51401a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51401075e0 0x7f5140199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f512c07ae10 0x7f512c07d2d0 unknown :-1 s=CLOSED pgs=3917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 --2- 172.21.15.43:0/59055804 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5140101be0 0x7f51401994e0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.068+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 >> 172.21.15.43:0/59055804 conn(0x7f51400fd640 msgr2=0x7f51400ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.069+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 shutdown_connections 2024-09-13T13:46:15.070 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.069+0000 7f5144ccd640 1 -- 172.21.15.43:0/59055804 wait complete. 2024-09-13T13:46:15.081 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 8ef39e5e-57a7-4748-b29c-009db0c54857 max_objects 10 2024-09-13T13:46:15.176 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 -- 172.21.15.43:0/257651661 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc104520 msgr2=0x7f0efc10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:15.176 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 --2- 172.21.15.43:0/257651661 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc104520 0x7f0efc10ca00 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009900 tx=0x7f0ef002a410 comp rx=0 tx=0).stop 2024-09-13T13:46:15.176 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 -- 172.21.15.43:0/257651661 shutdown_connections 2024-09-13T13:46:15.176 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 --2- 172.21.15.43:0/257651661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc10d050 0x7f0efc10f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.176 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 --2- 172.21.15.43:0/257651661 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc104520 0x7f0efc10ca00 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.176 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 --2- 172.21.15.43:0/257651661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc103c00 0x7f0efc103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 -- 172.21.15.43:0/257651661 >> 172.21.15.43:0/257651661 conn(0x7f0efc0fd630 msgr2=0x7f0efc0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 -- 172.21.15.43:0/257651661 shutdown_connections 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.172+0000 7f0f01f89640 1 -- 172.21.15.43:0/257651661 wait complete. 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 Processor -- start 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 -- start start 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc103c00 0x7f0efc1018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc104520 0x7f0efc101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc10d050 0x7f0efc102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1121b0 con 0x7f0efc103c00 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc112030 con 0x7f0efc104520 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0f01f89640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc112330 con 0x7f0efc10d050 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc103c00 0x7f0efc1018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:15.177 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc10d050 0x7f0efc102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0efb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc103c00 0x7f0efc1018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56700/0 (socket says 172.21.15.43:56700) 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0efb7fe640 1 -- 172.21.15.43:0/1124299138 learned_addr learned my addr 172.21.15.43:0/1124299138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0efbfff640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc10d050 0x7f0efc102320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:40538/0 (socket says 172.21.15.43:40538) 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.173+0000 7f0efaffd640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc104520 0x7f0efc101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efbfff640 1 -- 172.21.15.43:0/1124299138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc104520 msgr2=0x7f0efc101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efbfff640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc104520 0x7f0efc101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efbfff640 1 -- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc103c00 msgr2=0x7f0efc1018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efbfff640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc103c00 0x7f0efc1018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efbfff640 1 -- 172.21.15.43:0/1124299138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc1a6c00 con 0x7f0efc10d050 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efb7fe640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc103c00 0x7f0efc1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efaffd640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc104520 0x7f0efc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0efbfff640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc10d050 0x7f0efc102320 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400ec50 tx=0x7f0ef400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4003b70 con 0x7f0efc10d050 2024-09-13T13:46:15.178 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0ef4003d10 con 0x7f0efc10d050 2024-09-13T13:46:15.179 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc1a6ef0 con 0x7f0efc10d050 2024-09-13T13:46:15.179 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4010690 con 0x7f0efc10d050 2024-09-13T13:46:15.179 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.174+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc1a7360 con 0x7f0efc10d050 2024-09-13T13:46:15.179 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.175+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0efc103fe0 con 0x7f0efc10d050 2024-09-13T13:46:15.180 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.176+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f0ef4003fc0 con 0x7f0efc10d050 2024-09-13T13:46:15.180 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.176+0000 7f0ef8ff9640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f0ee407ae10 0x7f0ee407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:46:15.180 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.177+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(755..755 src has 1..755) ==== 9351+0+0 (secure 0 0 0) 0x7f0ef409f180 con 0x7f0efc10d050 2024-09-13T13:46:15.180 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.177+0000 7f0efb7fe640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f0ee407ae10 0x7f0ee407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:46:15.180 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.177+0000 7f0efb7fe640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f0ee407ae10 0x7f0ee407d2d0 secure :-1 s=READY pgs=3918 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0104e0 tx=0x7f0eec00b410 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:46:15.182 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.181+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f0ef4069c50 con 0x7f0efc10d050 2024-09-13T13:46:15.370 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:15.369+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} v 0) -- 0x7f0efc102ba0 con 0x7f0efc10d050 2024-09-13T13:46:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]': finished 2024-09-13T13:46:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:15 smithi189 ceph-mon[32912]: osdmap e754: 8 total, 8 up, 8 in 2024-09-13T13:46:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:15 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/59055804' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:15 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:15.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:15 smithi189 ceph-mon[32912]: pgmap v1194: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:46:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]': finished 2024-09-13T13:46:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[40255]: osdmap e754: 8 total, 8 up, 8 in 2024-09-13T13:46:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/59055804' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[40255]: pgmap v1194: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:46:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]': finished 2024-09-13T13:46:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[30106]: osdmap e754: 8 total, 8 up, 8 in 2024-09-13T13:46:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/59055804' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:15.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"} : dispatch 2024-09-13T13:46:15.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-mon[30106]: pgmap v1194: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-09-13T13:46:15.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:46:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:46:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:46:16.068 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:16.066+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 8ef39e5e-57a7-4748-b29c-009db0c54857 v756) ==== 223+0+0 (secure 0 0 0) 0x7f0ef406eb00 con 0x7f0efc10d050 2024-09-13T13:46:16.186 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:16.185+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} v 0) -- 0x7f0efc107c10 con 0x7f0efc10d050 2024-09-13T13:46:16.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:46:16 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:46:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]': finished 2024-09-13T13:46:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:16 smithi189 ceph-mon[32912]: osdmap e755: 8 total, 8 up, 8 in 2024-09-13T13:46:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:16 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1124299138' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:16.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:16 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:16.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]': finished 2024-09-13T13:46:16.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[40255]: osdmap e755: 8 total, 8 up, 8 in 2024-09-13T13:46:16.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1124299138' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "app": "rados"}]': finished 2024-09-13T13:46:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[30106]: osdmap e755: 8 total, 8 up, 8 in 2024-09-13T13:46:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1124299138' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:16 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:17.081 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.080+0000 7f0ef8ff9640 1 -- 172.21.15.43:0/1124299138 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 8ef39e5e-57a7-4748-b29c-009db0c54857 v757) ==== 223+0+0 (secure 0 0 0) 0x7f0ef406c120 con 0x7f0efc10d050 2024-09-13T13:46:17.081 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool 8ef39e5e-57a7-4748-b29c-009db0c54857 2024-09-13T13:46:17.084 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f0ee407ae10 msgr2=0x7f0ee407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:17.084 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f0ee407ae10 0x7f0ee407d2d0 secure :-1 s=READY pgs=3918 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0104e0 tx=0x7f0eec00b410 comp rx=0 tx=0).stop 2024-09-13T13:46:17.084 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc10d050 msgr2=0x7f0efc102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc10d050 0x7f0efc102320 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400ec50 tx=0x7f0ef400c6a0 comp rx=0 tx=0).stop 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 shutdown_connections 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f0ee407ae10 0x7f0ee407d2d0 unknown :-1 s=CLOSED pgs=3918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0efc10d050 0x7f0efc102320 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc104520 0x7f0efc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 --2- 172.21.15.43:0/1124299138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0efc103c00 0x7f0efc1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 >> 172.21.15.43:0/1124299138 conn(0x7f0efc0fd630 msgr2=0x7f0efc104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 shutdown_connections 2024-09-13T13:46:17.085 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:46:17.083+0000 7f0f01f89640 1 -- 172.21.15.43:0/1124299138 wait complete. 2024-09-13T13:46:17.095 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-13T13:46:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:17 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:46:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:17 smithi189 ceph-mon[32912]: osdmap e756: 8 total, 8 up, 8 in 2024-09-13T13:46:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:17 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/1124299138' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:17 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:17.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:17 smithi189 ceph-mon[32912]: pgmap v1197: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-13T13:46:17.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[40255]: osdmap e756: 8 total, 8 up, 8 in 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/1124299138' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[40255]: pgmap v1197: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[30106]: osdmap e756: 8 total, 8 up, 8 in 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/1124299138' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"} : dispatch 2024-09-13T13:46:17.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:17 smithi043 ceph-mon[30106]: pgmap v1197: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-13T13:46:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:46:18.398 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:18 smithi189 ceph-mon[32912]: osdmap e757: 8 total, 8 up, 8 in 2024-09-13T13:46:18.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:46:18.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:18 smithi043 ceph-mon[30106]: osdmap e757: 8 total, 8 up, 8 in 2024-09-13T13:46:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "field": "max_objects", "val": "10"}]': finished 2024-09-13T13:46:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:18 smithi043 ceph-mon[40255]: osdmap e757: 8 total, 8 up, 8 in 2024-09-13T13:46:19.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:19 smithi189 ceph-mon[32912]: pgmap v1199: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.9 KiB/s wr, 1 op/s 2024-09-13T13:46:19.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:19 smithi043 ceph-mon[30106]: pgmap v1199: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.9 KiB/s wr, 1 op/s 2024-09-13T13:46:19.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:19 smithi043 ceph-mon[40255]: pgmap v1199: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.9 KiB/s wr, 1 op/s 2024-09-13T13:46:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:21 smithi189 ceph-mon[32912]: pgmap v1200: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-13T13:46:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:21 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:46:21.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:21 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:21 smithi043 ceph-mon[40255]: pgmap v1200: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-13T13:46:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:21 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:46:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:21 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:21 smithi043 ceph-mon[30106]: pgmap v1200: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-13T13:46:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:21 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:46:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:21 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:23.900 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:23 smithi189 ceph-mon[32912]: pgmap v1201: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-09-13T13:46:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:23 smithi043 ceph-mon[30106]: pgmap v1201: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-09-13T13:46:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:23 smithi043 ceph-mon[40255]: pgmap v1201: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-09-13T13:46:25.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:25 smithi189 ceph-mon[32912]: pgmap v1202: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 607 B/s rd, 0 op/s 2024-09-13T13:46:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:25 smithi043 ceph-mon[30106]: pgmap v1202: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 607 B/s rd, 0 op/s 2024-09-13T13:46:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:25 smithi043 ceph-mon[40255]: pgmap v1202: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 607 B/s rd, 0 op/s 2024-09-13T13:46:25.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:46:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:46:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:46:26.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:46:26 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:46:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:27 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:27.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:27 smithi189 ceph-mon[32912]: pgmap v1203: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:46:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:27 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:27 smithi043 ceph-mon[30106]: pgmap v1203: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:46:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:27 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:27.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:27 smithi043 ceph-mon[40255]: pgmap v1203: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:46:29.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:29 smithi189 ceph-mon[32912]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 896 B/s rd, 0 op/s 2024-09-13T13:46:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:29 smithi043 ceph-mon[30106]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 896 B/s rd, 0 op/s 2024-09-13T13:46:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:29 smithi043 ceph-mon[40255]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 896 B/s rd, 0 op/s 2024-09-13T13:46:31.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:31 smithi189 ceph-mon[32912]: pgmap v1205: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:31 smithi043 ceph-mon[40255]: pgmap v1205: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:31 smithi043 ceph-mon[30106]: pgmap v1205: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:33.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:33 smithi189 ceph-mon[32912]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:33 smithi043 ceph-mon[30106]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:33.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:33 smithi043 ceph-mon[40255]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:35 smithi189 ceph-mon[32912]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:35.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:35 smithi043 ceph-mon[30106]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:35 smithi043 ceph-mon[40255]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:35.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:46:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:46:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:46:36.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:46:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:46:37.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:37 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:37.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:37 smithi043 ceph-mon[40255]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:37 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:37 smithi043 ceph-mon[30106]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:37 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:37 smithi189 ceph-mon[32912]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:39.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:39 smithi043 ceph-mon[30106]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:39 smithi043 ceph-mon[40255]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:39 smithi189 ceph-mon[32912]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:41.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:41 smithi043 ceph-mon[30106]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:41.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:41 smithi043 ceph-mon[40255]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:41 smithi189 ceph-mon[32912]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:43 smithi043 ceph-mon[30106]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:43 smithi043 ceph-mon[40255]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:43 smithi189 ceph-mon[32912]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:45 smithi043 ceph-mon[30106]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:45 smithi043 ceph-mon[40255]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:45.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:46:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:46:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:46:46.076 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:45 smithi189 ceph-mon[32912]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:46.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:46:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:46:47.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-09-13T13:46:47.098 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.098 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj1 /etc/passwd 2024-09-13T13:46:47.142 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.142 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj2 /etc/passwd 2024-09-13T13:46:47.182 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.182 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj3 /etc/passwd 2024-09-13T13:46:47.221 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.221 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj4 /etc/passwd 2024-09-13T13:46:47.258 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.258 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj5 /etc/passwd 2024-09-13T13:46:47.295 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.295 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj6 /etc/passwd 2024-09-13T13:46:47.332 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.332 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj7 /etc/passwd 2024-09-13T13:46:47.369 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.369 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj8 /etc/passwd 2024-09-13T13:46:47.406 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.406 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj9 /etc/passwd 2024-09-13T13:46:47.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-09-13T13:46:47.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 8ef39e5e-57a7-4748-b29c-009db0c54857 put obj10 /etc/passwd 2024-09-13T13:46:47.479 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-13T13:46:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:47 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:47 smithi043 ceph-mon[40255]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:47 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:47 smithi043 ceph-mon[30106]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:47 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:47 smithi189 ceph-mon[32912]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:46:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:49 smithi043 ceph-mon[30106]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:49 smithi043 ceph-mon[40255]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:49 smithi189 ceph-mon[32912]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:46:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:51.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:46:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:51 smithi043 ceph-mon[30106]: pgmap v1215: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-13T13:46:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:51 smithi043 ceph-mon[40255]: pgmap v1215: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-13T13:46:52.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:51 smithi189 ceph-mon[32912]: pgmap v1215: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-13T13:46:52.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:52 smithi043 ceph-mon[30106]: pool '8ef39e5e-57a7-4748-b29c-009db0c54857' is full (reached quota's max_objects: 10) 2024-09-13T13:46:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:52 smithi043 ceph-mon[30106]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:46:52.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:52 smithi043 ceph-mon[30106]: osdmap e758: 8 total, 8 up, 8 in 2024-09-13T13:46:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:52 smithi043 ceph-mon[40255]: pool '8ef39e5e-57a7-4748-b29c-009db0c54857' is full (reached quota's max_objects: 10) 2024-09-13T13:46:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:52 smithi043 ceph-mon[40255]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:46:52.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:52 smithi043 ceph-mon[40255]: osdmap e758: 8 total, 8 up, 8 in 2024-09-13T13:46:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:52 smithi189 ceph-mon[32912]: pool '8ef39e5e-57a7-4748-b29c-009db0c54857' is full (reached quota's max_objects: 10) 2024-09-13T13:46:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:52 smithi189 ceph-mon[32912]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-13T13:46:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:52 smithi189 ceph-mon[32912]: osdmap e758: 8 total, 8 up, 8 in 2024-09-13T13:46:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:53 smithi043 ceph-mon[30106]: pgmap v1217: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:53 smithi043 ceph-mon[40255]: pgmap v1217: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:53 smithi189 ceph-mon[32912]: pgmap v1217: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:55.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:55 smithi043 ceph-mon[30106]: pgmap v1218: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:55.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:46:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:46:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:46:55.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:55 smithi043 ceph-mon[40255]: pgmap v1218: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:56.078 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:55 smithi189 ceph-mon[32912]: pgmap v1218: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:56.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:46:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:46:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:57 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:57.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:57 smithi043 ceph-mon[30106]: pgmap v1219: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:57 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:57.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:57 smithi043 ceph-mon[40255]: pgmap v1219: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:57 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:46:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:57 smithi189 ceph-mon[32912]: pgmap v1219: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:46:59 smithi043 ceph-mon[30106]: pgmap v1220: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:46:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:46:59 smithi043 ceph-mon[40255]: pgmap v1220: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:47:00.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:46:59 smithi189 ceph-mon[32912]: pgmap v1220: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-13T13:47:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[30106]: pgmap v1221: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[40255]: pgmap v1221: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:47:01.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:01 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:47:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:01 smithi189 ceph-mon[32912]: pgmap v1221: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-13T13:47:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:01 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T13:47:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:01 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T13:47:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:01 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T13:47:02.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:01 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:47:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:03 smithi043 ceph-mon[30106]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-09-13T13:47:03.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:03 smithi043 ceph-mon[40255]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-09-13T13:47:04.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:03 smithi189 ceph-mon[32912]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-09-13T13:47:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:05 smithi043 ceph-mon[30106]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:05 smithi043 ceph-mon[30106]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:47:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:05 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:05.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:05 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:47:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:47:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:05 smithi043 ceph-mon[40255]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:05 smithi043 ceph-mon[40255]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:47:05.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:05 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:06.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:05 smithi189 ceph-mon[32912]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:06.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:05 smithi189 ceph-mon[32912]: from='mgr.24458 ' entity='mgr.y' 2024-09-13T13:47:06.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:05 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:06.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:47:06 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:47:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:07 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:07 smithi043 ceph-mon[30106]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:07 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:07.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:07 smithi043 ceph-mon[40255]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:07 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:08.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:07 smithi189 ceph-mon[32912]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:09 smithi043 ceph-mon[30106]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:09.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:09 smithi043 ceph-mon[40255]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:10.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:09 smithi189 ceph-mon[32912]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:11 smithi043 ceph-mon[30106]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:11.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:11 smithi043 ceph-mon[40255]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:12.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:11 smithi189 ceph-mon[32912]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:13 smithi043 ceph-mon[30106]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:13 smithi043 ceph-mon[40255]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:14.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:13 smithi189 ceph-mon[32912]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:15.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:15 smithi043 ceph-mon[40255]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:15 smithi043 ceph-mon[30106]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:15.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:15 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:47:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:47:16.095 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:15 smithi189 ceph-mon[32912]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:16.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:47:16 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:47:17.480 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p 002fcfa4-dfdd-44b5-b965-f404930ab85a put threemore /etc/passwd 2024-09-13T13:47:17.523 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 002fcfa4-dfdd-44b5-b965-f404930ab85a max_bytes 0 2024-09-13T13:47:17.617 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 -- 172.21.15.43:0/4269295632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 msgr2=0x7fd39010e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 --2- 172.21.15.43:0/4269295632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 0x7fd39010e770 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7fd388009ca0 tx=0x7fd38802a8c0 comp rx=0 tx=0).stop 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 -- 172.21.15.43:0/4269295632 shutdown_connections 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 --2- 172.21.15.43:0/4269295632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 0x7fd39010e770 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 --2- 172.21.15.43:0/4269295632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3901075f0 0x7fd390107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 --2- 172.21.15.43:0/4269295632 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd390101bf0 0x7fd390101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 -- 172.21.15.43:0/4269295632 >> 172.21.15.43:0/4269295632 conn(0x7fd3900fd630 msgr2=0x7fd3900ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 -- 172.21.15.43:0/4269295632 shutdown_connections 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 -- 172.21.15.43:0/4269295632 wait complete. 2024-09-13T13:47:17.618 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 Processor -- start 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.614+0000 7fd3982bd640 1 -- start start 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd390101bf0 0x7fd3901034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3901075f0 0x7fd3901039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 0x7fd390103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901110c0 con 0x7fd390107f70 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd390110f40 con 0x7fd390101bf0 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7fd390111240 con 0x7fd3901075f0 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3901075f0 0x7fd3901039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd39685a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 0x7fd390103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3901075f0 0x7fd3901039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:41508/0 (socket says 172.21.15.43:41508) 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 -- 172.21.15.43:0/492049025 learned_addr learned my addr 172.21.15.43:0/492049025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd396059640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd390101bf0 0x7fd3901034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:17.619 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 -- 172.21.15.43:0/492049025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd390101bf0 msgr2=0x7fd3901034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd390101bf0 0x7fd3901034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 -- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 msgr2=0x7fd390103f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 0x7fd390103f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3901a6ae0 con 0x7fd3901075f0 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd395858640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3901075f0 0x7fd3901039e0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fd38000d950 tx=0x7fd38000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380017070 con 0x7fd3901075f0 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fd380009c30 con 0x7fd3901075f0 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3901a6dd0 con 0x7fd3901075f0 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38000f3c0 con 0x7fd3901075f0 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.615+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3901a72e0 con 0x7fd3901075f0 2024-09-13T13:47:17.620 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.616+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd390101fd0 con 0x7fd3901075f0 2024-09-13T13:47:17.621 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.617+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7fd380010460 con 0x7fd3901075f0 2024-09-13T13:47:17.621 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.618+0000 7fd3877fe640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fd37c07ae10 0x7fd37c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:17.621 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.618+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(758..758 src has 1..758) ==== 9351+0+0 (secure 0 0 0) 0x7fd38009e780 con 0x7fd3901075f0 2024-09-13T13:47:17.621 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.618+0000 7fd396059640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fd37c07ae10 0x7fd37c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:17.622 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.618+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=759}) -- 0x7fd37c086fa0 con 0x7fd3901075f0 2024-09-13T13:47:17.622 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.618+0000 7fd396059640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fd37c07ae10 0x7fd37c07d2d0 secure :-1 s=READY pgs=3930 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0095f0 tx=0x7fd38c005f90 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:17.623 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.622+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fd380069270 con 0x7fd3901075f0 2024-09-13T13:47:17.811 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:17.810+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} v 0) -- 0x7fd3901a7510 con 0x7fd3901075f0 2024-09-13T13:47:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:17 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:17 smithi043 ceph-mon[30106]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:17 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:17 smithi043 ceph-mon[40255]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:18.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:17 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:18.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:17 smithi189 ceph-mon[32912]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:18.719 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:18.718+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(759..759 src has 1..759) ==== 628+0+0 (secure 0 0 0) 0x7fd3800d5190 con 0x7fd3901075f0 2024-09-13T13:47:18.719 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:18.718+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=760}) -- 0x7fd37c087b50 con 0x7fd3901075f0 2024-09-13T13:47:18.723 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:18.722+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v759) ==== 217+0+0 (secure 0 0 0) 0x7fd38006e120 con 0x7fd3901075f0 2024-09-13T13:47:18.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:18.839+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} v 0) -- 0x7fd39010a7f0 con 0x7fd3901075f0 2024-09-13T13:47:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:18 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/492049025' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:18 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:18 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/492049025' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:18.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:18 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:19.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:18 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/492049025' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:19.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:18 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:19.729 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.728+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 9 ==== osd_map(760..760 src has 1..760) ==== 628+0+0 (secure 0 0 0) 0x7fd3800d5190 con 0x7fd3901075f0 2024-09-13T13:47:19.729 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.728+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=761}) -- 0x7fd37c088150 con 0x7fd3901075f0 2024-09-13T13:47:19.734 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.733+0000 7fd3877fe640 1 -- 172.21.15.43:0/492049025 <== mon.2 v2:172.21.15.43:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v760) ==== 217+0+0 (secure 0 0 0) 0x7fd38009bed0 con 0x7fd3901075f0 2024-09-13T13:47:19.734 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fd37c07ae10 msgr2=0x7fd37c07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fd37c07ae10 0x7fd37c07d2d0 secure :-1 s=READY pgs=3930 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0095f0 tx=0x7fd38c005f90 comp rx=0 tx=0).stop 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3901075f0 msgr2=0x7fd3901039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3901075f0 0x7fd3901039e0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fd38000d950 tx=0x7fd38000de10 comp rx=0 tx=0).stop 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 shutdown_connections 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd390107f70 0x7fd390103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7fd37c07ae10 0x7fd37c07d2d0 unknown :-1 s=CLOSED pgs=3930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.738 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3901075f0 0x7fd3901039e0 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.739 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.736+0000 7fd3982bd640 1 --2- 172.21.15.43:0/492049025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd390101bf0 0x7fd3901034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.739 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.737+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 >> 172.21.15.43:0/492049025 conn(0x7fd3900fd630 msgr2=0x7fd39006b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:19.739 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.737+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 shutdown_connections 2024-09-13T13:47:19.739 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.737+0000 7fd3982bd640 1 -- 172.21.15.43:0/492049025 wait complete. 2024-09-13T13:47:19.749 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota 002fcfa4-dfdd-44b5-b965-f404930ab85a max_objects 0 2024-09-13T13:47:19.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 -- 172.21.15.43:0/28536645 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c06a540 msgr2=0x7f5f2c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:19.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 --2- 172.21.15.43:0/28536645 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c06a540 0x7f5f2c06a920 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009ca0 tx=0x7f5f1c02a8c0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 -- 172.21.15.43:0/28536645 shutdown_connections 2024-09-13T13:47:19.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 --2- 172.21.15.43:0/28536645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f2c109050 0x7f5f2c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 --2- 172.21.15.43:0/28536645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 0x7f5f2c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 --2- 172.21.15.43:0/28536645 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c06a540 0x7f5f2c06a920 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.840 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 -- 172.21.15.43:0/28536645 >> 172.21.15.43:0/28536645 conn(0x7f5f2c0776a0 msgr2=0x7f5f2c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:19.841 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 -- 172.21.15.43:0/28536645 shutdown_connections 2024-09-13T13:47:19.841 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 -- 172.21.15.43:0/28536645 wait complete. 2024-09-13T13:47:19.841 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 Processor -- start 2024-09-13T13:47:19.843 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.839+0000 7f5f317ec640 1 -- start start 2024-09-13T13:47:19.843 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f317ec640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f2c06a540 0x7f5f2c101e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:19.843 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f317ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 0x7f5f2c102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:19.843 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f317ec640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c109050 0x7f5f2c1028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:19.843 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f317ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1121e0 con 0x7f5f2c1086d0 2024-09-13T13:47:19.843 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f317ec640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c112060 con 0x7f5f2c06a540 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f317ec640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2c112360 con 0x7f5f2c109050 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 0x7f5f2c102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2b7fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c109050 0x7f5f2c1028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 0x7f5f2c102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36634/0 (socket says 172.21.15.43:36634) 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 -- 172.21.15.43:0/135120686 learned_addr learned my addr 172.21.15.43:0/135120686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2affd640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f2c06a540 0x7f5f2c101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 -- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c109050 msgr2=0x7f5f2c1028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c109050 0x7f5f2c1028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 -- 172.21.15.43:0/135120686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f2c06a540 msgr2=0x7f5f2c101e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f2c06a540 0x7f5f2c101e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:19.844 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c1a6c10 con 0x7f5f2c1086d0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2affd640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f2c06a540 0x7f5f2c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f2a7fc640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 0x7f5f2c102380 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000ea60 tx=0x7f5f2000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000cde0 con 0x7f5f2c1086d0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f5f20004470 con 0x7f5f2c1086d0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c1a6f00 con 0x7f5f2c1086d0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.840+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20010430 con 0x7f5f2c1086d0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.841+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c1a73c0 con 0x7f5f2c1086d0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.842+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c06a920 con 0x7f5f2c1086d0 2024-09-13T13:47:19.845 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.842+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f5f20003f60 con 0x7f5f2c1086d0 2024-09-13T13:47:19.846 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.843+0000 7f5f13fff640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f5f1407ae10 0x7f5f1407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:19.846 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.843+0000 7f5f2affd640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f5f1407ae10 0x7f5f1407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:19.846 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.843+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(760..760 src has 1..760) ==== 9351+0+0 (secure 0 0 0) 0x7f5f2009ef20 con 0x7f5f2c1086d0 2024-09-13T13:47:19.846 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.843+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=761}) -- 0x7f5f14086f20 con 0x7f5f2c1086d0 2024-09-13T13:47:19.846 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.843+0000 7f5f2affd640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f5f1407ae10 0x7f5f1407d2d0 secure :-1 s=READY pgs=3931 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c031fd0 tx=0x7f5f1c005c30 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:19.848 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:19.847+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f5f200699f0 con 0x7f5f2c1086d0 2024-09-13T13:47:19.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[40255]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:19.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:47:19.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[40255]: osdmap e759: 8 total, 8 up, 8 in 2024-09-13T13:47:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/492049025' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[30106]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:47:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[30106]: osdmap e759: 8 total, 8 up, 8 in 2024-09-13T13:47:19.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/492049025' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:19.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:19 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:20.040 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:20.039+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} v 0) -- 0x7f5f2c02e5c0 con 0x7f5f2c1086d0 2024-09-13T13:47:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:19 smithi189 ceph-mon[32912]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:47:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:19 smithi189 ceph-mon[32912]: osdmap e759: 8 total, 8 up, 8 in 2024-09-13T13:47:20.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:19 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/492049025' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:20.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:19 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-13T13:47:20.734 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:20.733+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v761) ==== 221+0+0 (secure 0 0 0) 0x7f5f2006e8a0 con 0x7f5f2c1086d0 2024-09-13T13:47:20.734 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:20.733+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(761..761 src has 1..761) ==== 628+0+0 (secure 0 0 0) 0x7f5f20004250 con 0x7f5f2c1086d0 2024-09-13T13:47:20.734 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:20.733+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=762}) -- 0x7f5f14087ec0 con 0x7f5f2c1086d0 2024-09-13T13:47:20.852 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:20.851+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} v 0) -- 0x7f5f2c02e810 con 0x7f5f2c1086d0 2024-09-13T13:47:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:47:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[30106]: osdmap e760: 8 total, 8 up, 8 in 2024-09-13T13:47:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:47:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:47:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[40255]: osdmap e760: 8 total, 8 up, 8 in 2024-09-13T13:47:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:47:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:20 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:21.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:20 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_bytes", "val": "0"}]': finished 2024-09-13T13:47:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:20 smithi189 ceph-mon[32912]: osdmap e760: 8 total, 8 up, 8 in 2024-09-13T13:47:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:20 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:47:21.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:20 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:21.740 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.738+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a v762) ==== 221+0+0 (secure 0 0 0) 0x7f5f2009c6a0 con 0x7f5f2c1086d0 2024-09-13T13:47:21.740 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool 002fcfa4-dfdd-44b5-b965-f404930ab85a 2024-09-13T13:47:21.740 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.739+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 <== mon.0 v2:172.21.15.43:3300/0 10 ==== osd_map(762..762 src has 1..762) ==== 628+0+0 (secure 0 0 0) 0x7f5f20004250 con 0x7f5f2c1086d0 2024-09-13T13:47:21.740 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.739+0000 7f5f13fff640 1 -- 172.21.15.43:0/135120686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=763}) -- 0x7f5f14088180 con 0x7f5f2c1086d0 2024-09-13T13:47:21.745 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f5f1407ae10 msgr2=0x7f5f1407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:21.745 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f5f1407ae10 0x7f5f1407d2d0 secure :-1 s=READY pgs=3931 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c031fd0 tx=0x7f5f1c005c30 comp rx=0 tx=0).stop 2024-09-13T13:47:21.745 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 msgr2=0x7f5f2c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 0x7f5f2c102380 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000ea60 tx=0x7f5f2000ef20 comp rx=0 tx=0).stop 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 shutdown_connections 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f5f1407ae10 0x7f5f1407d2d0 unknown :-1 s=CLOSED pgs=3931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f2c109050 0x7f5f2c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c1086d0 0x7f5f2c102380 secure :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000ea60 tx=0x7f5f2000ef20 comp rx=0 tx=0).stop 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 --2- 172.21.15.43:0/135120686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f2c06a540 0x7f5f2c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.744+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 >> 172.21.15.43:0/135120686 conn(0x7f5f2c0776a0 msgr2=0x7f5f2c104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:21.746 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.745+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 shutdown_connections 2024-09-13T13:47:21.747 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:21.746+0000 7f5f317ec640 1 -- 172.21.15.43:0/135120686 wait complete. 2024-09-13T13:47:21.758 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-09-13T13:47:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[30106]: pgmap v1233: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-13T13:47:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:47:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[30106]: osdmap e761: 8 total, 8 up, 8 in 2024-09-13T13:47:21.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:47:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[40255]: pgmap v1233: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-13T13:47:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:47:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[40255]: osdmap e761: 8 total, 8 up, 8 in 2024-09-13T13:47:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:21 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:47:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:21 smithi189 ceph-mon[32912]: pgmap v1233: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-13T13:47:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:47:22.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:21 smithi189 ceph-mon[32912]: osdmap e761: 8 total, 8 up, 8 in 2024-09-13T13:47:22.149 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:21 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"} : dispatch 2024-09-13T13:47:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:22 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:47:23.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:22 smithi189 ceph-mon[32912]: osdmap e762: 8 total, 8 up, 8 in 2024-09-13T13:47:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:22 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:47:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:22 smithi043 ceph-mon[30106]: osdmap e762: 8 total, 8 up, 8 in 2024-09-13T13:47:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:22 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/135120686' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "field": "max_objects", "val": "0"}]': finished 2024-09-13T13:47:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:22 smithi043 ceph-mon[40255]: osdmap e762: 8 total, 8 up, 8 in 2024-09-13T13:47:24.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:23 smithi189 ceph-mon[32912]: pgmap v1236: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-13T13:47:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:23 smithi043 ceph-mon[30106]: pgmap v1236: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-13T13:47:24.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:23 smithi043 ceph-mon[40255]: pgmap v1236: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-13T13:47:25.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:25 smithi043 ceph-mon[30106]: pgmap v1237: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 882 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-13T13:47:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:25 smithi043 ceph-mon[40255]: pgmap v1237: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 882 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-13T13:47:25.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:47:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:47:26.096 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:25 smithi189 ceph-mon[32912]: pgmap v1237: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 882 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-13T13:47:26.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:47:26 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:47:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:27 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:28.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:27 smithi189 ceph-mon[32912]: pgmap v1238: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-13T13:47:28.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:27 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:28.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:27 smithi043 ceph-mon[30106]: pgmap v1238: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-13T13:47:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:27 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:27 smithi043 ceph-mon[40255]: pgmap v1238: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-09-13T13:47:30.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:29 smithi189 ceph-mon[32912]: pgmap v1239: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:47:30.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:29 smithi043 ceph-mon[30106]: pgmap v1239: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:47:30.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:29 smithi043 ceph-mon[40255]: pgmap v1239: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-13T13:47:32.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:31 smithi189 ceph-mon[32912]: pgmap v1240: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:47:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:31 smithi043 ceph-mon[30106]: pgmap v1240: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:47:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:31 smithi043 ceph-mon[40255]: pgmap v1240: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T13:47:34.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:33 smithi189 ceph-mon[32912]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 949 B/s rd, 0 op/s 2024-09-13T13:47:34.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:33 smithi043 ceph-mon[30106]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 949 B/s rd, 0 op/s 2024-09-13T13:47:34.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:33 smithi043 ceph-mon[40255]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 949 B/s rd, 0 op/s 2024-09-13T13:47:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:35 smithi043 ceph-mon[30106]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:35 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:35.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:35 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:47:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:47:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:35 smithi043 ceph-mon[40255]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:35 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:36.103 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:35 smithi189 ceph-mon[32912]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:36.103 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:35 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:36.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:47:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:47:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:37 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:38.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:37 smithi189 ceph-mon[32912]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:38.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:37 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:38.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:37 smithi043 ceph-mon[30106]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:38.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:37 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:38.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:37 smithi043 ceph-mon[40255]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:40.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:39 smithi189 ceph-mon[32912]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:40.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:39 smithi043 ceph-mon[30106]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:39 smithi043 ceph-mon[40255]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:42.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:41 smithi189 ceph-mon[32912]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:42.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:41 smithi043 ceph-mon[30106]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:42.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:41 smithi043 ceph-mon[40255]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:44.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:43 smithi189 ceph-mon[32912]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:44.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:43 smithi043 ceph-mon[30106]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:44.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:43 smithi043 ceph-mon[40255]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:45 smithi043 ceph-mon[30106]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:45 smithi043 ceph-mon[40255]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:45.992 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:45 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:47:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:47:46.114 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:45 smithi189 ceph-mon[32912]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:46.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:47:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:47:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:47 smithi189 ceph-mon[32912]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:48.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:47 smithi189 ceph-mon[32912]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:48.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:47 smithi043 ceph-mon[30106]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:48.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:47 smithi043 ceph-mon[30106]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:47 smithi043 ceph-mon[40255]: from='client.24383 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-13T13:47:48.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:47 smithi043 ceph-mon[40255]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:50.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:49 smithi189 ceph-mon[32912]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:50.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:49 smithi043 ceph-mon[30106]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:49 smithi043 ceph-mon[40255]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:51.049 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:50 smithi189 ceph-mon[32912]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:51.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:50 smithi043 ceph-mon[30106]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:51.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:50 smithi043 ceph-mon[40255]: from='mgr.24458 172.21.15.43:0/240149779' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-13T13:47:51.760 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete 002fcfa4-dfdd-44b5-b965-f404930ab85a 002fcfa4-dfdd-44b5-b965-f404930ab85a --yes-i-really-really-mean-it 2024-09-13T13:47:51.862 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/381050421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0107f60 msgr2=0x7ff0f010e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:51.863 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/381050421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0107f60 0x7ff0f010e760 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009ca0 tx=0x7ff0e802a8c0 comp rx=0 tx=0).stop 2024-09-13T13:47:51.863 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/381050421 shutdown_connections 2024-09-13T13:47:51.863 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/381050421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0107f60 0x7ff0f010e760 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:51.863 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/381050421 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 0x7ff0f0107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:51.863 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/381050421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0f0101be0 0x7ff0f0101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:51.864 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/381050421 >> 172.21.15.43:0/381050421 conn(0x7ff0f00fd640 msgr2=0x7ff0f00ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:51.864 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/381050421 shutdown_connections 2024-09-13T13:47:51.864 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.856+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/381050421 wait complete. 2024-09-13T13:47:51.864 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 Processor -- start 2024-09-13T13:47:51.864 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 -- start start 2024-09-13T13:47:51.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0101be0 0x7ff0f0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:51.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 0x7ff0f0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:51.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0f0107f60 0x7ff0f019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:51.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01110f0 con 0x7ff0f0101be0 2024-09-13T13:47:51.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0110f70 con 0x7ff0f0107f60 2024-09-13T13:47:51.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f7e2a640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f0111270 con 0x7ff0f01075e0 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f5bc6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0101be0 0x7ff0f0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f53c5640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 0x7ff0f0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f5bc6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0101be0 0x7ff0f0199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41140/0 (socket says 172.21.15.43:41140) 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f53c5640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 0x7ff0f0199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:51618/0 (socket says 172.21.15.43:51618) 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f5bc6640 1 -- 172.21.15.43:0/2956339235 learned_addr learned my addr 172.21.15.43:0/2956339235 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f63c7640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0f0107f60 0x7ff0f019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f53c5640 1 -- 172.21.15.43:0/2956339235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0f0107f60 msgr2=0x7ff0f019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:51.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f53c5640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0f0107f60 0x7ff0f019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f53c5640 1 -- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0101be0 msgr2=0x7ff0f0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f53c5640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0101be0 0x7ff0f0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f53c5640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f01a4960 con 0x7ff0f01075e0 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.857+0000 7ff0f5bc6640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0101be0 0x7ff0f0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.858+0000 7ff0f53c5640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 0x7ff0f0199aa0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000e9f0 tx=0x7ff0e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.858+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000cd80 con 0x7ff0f01075e0 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.858+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f01a4c50 con 0x7ff0f01075e0 2024-09-13T13:47:51.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.858+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ff0e0004470 con 0x7ff0f01075e0 2024-09-13T13:47:51.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.858+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0003e10 con 0x7ff0f01075e0 2024-09-13T13:47:51.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.858+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01a51a0 con 0x7ff0f01075e0 2024-09-13T13:47:51.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.859+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f0101fc0 con 0x7ff0f01075e0 2024-09-13T13:47:51.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.860+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7ff0e0003fb0 con 0x7ff0f01075e0 2024-09-13T13:47:51.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.860+0000 7ff0e6ffd640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff0dc07ae10 0x7ff0dc07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:51.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.860+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(762..762 src has 1..762) ==== 9351+0+0 (secure 0 0 0) 0x7ff0e009edf0 con 0x7ff0f01075e0 2024-09-13T13:47:51.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.860+0000 7ff0f5bc6640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff0dc07ae10 0x7ff0dc07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:51.869 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.860+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=763}) -- 0x7ff0dc086f20 con 0x7ff0f01075e0 2024-09-13T13:47:51.869 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.861+0000 7ff0f5bc6640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff0dc07ae10 0x7ff0dc07d2d0 secure :-1 s=READY pgs=3932 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec011550 tx=0x7ff0ec005f90 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:51.870 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:51.865+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff0e00698c0 con 0x7ff0f01075e0 2024-09-13T13:47:52.054 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.053+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff0f0109d40 con 0x7ff0f01075e0 2024-09-13T13:47:52.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:51 smithi189 ceph-mon[32912]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:51 smithi043 ceph-mon[30106]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:51 smithi043 ceph-mon[40255]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-13T13:47:52.804 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.803+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(763..763 src has 1..763) ==== 296+0+0 (secure 0 0 0) 0x7ff0e0009420 con 0x7ff0f01075e0 2024-09-13T13:47:52.805 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.803+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=764}) -- 0x7ff0dc0877f0 con 0x7ff0f01075e0 2024-09-13T13:47:52.810 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.807+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true}]=0 pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' removed v763) ==== 248+0+0 (secure 0 0 0) 0x7ff0e006e770 con 0x7ff0f01075e0 2024-09-13T13:47:52.927 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.926+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff0f0109980 con 0x7ff0f01075e0 2024-09-13T13:47:52.928 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.927+0000 7ff0e6ffd640 1 -- 172.21.15.43:0/2956339235 <== mon.2 v2:172.21.15.43:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true}]=0 pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' does not exist v763) ==== 255+0+0 (secure 0 0 0) 0x7ff0e009c570 con 0x7ff0f01075e0 2024-09-13T13:47:52.928 INFO:tasks.workunit.client.0.smithi043.stderr:pool '002fcfa4-dfdd-44b5-b965-f404930ab85a' does not exist 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff0dc07ae10 msgr2=0x7ff0dc07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff0dc07ae10 0x7ff0dc07d2d0 secure :-1 s=READY pgs=3932 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec011550 tx=0x7ff0ec005f90 comp rx=0 tx=0).stop 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 msgr2=0x7ff0f0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 0x7ff0f0199aa0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000e9f0 tx=0x7ff0e000eeb0 comp rx=0 tx=0).stop 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 shutdown_connections 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0f0107f60 0x7ff0f019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7ff0dc07ae10 0x7ff0dc07d2d0 unknown :-1 s=CLOSED pgs=3932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0f01075e0 0x7ff0f0199aa0 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:52.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 --2- 172.21.15.43:0/2956339235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0f0101be0 0x7ff0f0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:52.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 >> 172.21.15.43:0/2956339235 conn(0x7ff0f00fd640 msgr2=0x7ff0f00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:52.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.929+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 shutdown_connections 2024-09-13T13:47:52.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:52.930+0000 7ff0f7e2a640 1 -- 172.21.15.43:0/2956339235 wait complete. 2024-09-13T13:47:52.939 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete 8ef39e5e-57a7-4748-b29c-009db0c54857 8ef39e5e-57a7-4748-b29c-009db0c54857 --yes-i-really-really-mean-it 2024-09-13T13:47:53.028 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 -- 172.21.15.43:0/709018385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724107f70 msgr2=0x7f372410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 --2- 172.21.15.43:0/709018385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724107f70 0x7f372410e770 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f371c009ca0 tx=0x7f371c02a8c0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 -- 172.21.15.43:0/709018385 shutdown_connections 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 --2- 172.21.15.43:0/709018385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724107f70 0x7f372410e770 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 --2- 172.21.15.43:0/709018385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 0x7f3724107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 --2- 172.21.15.43:0/709018385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3724101bf0 0x7f3724101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 -- 172.21.15.43:0/709018385 >> 172.21.15.43:0/709018385 conn(0x7f37240fd630 msgr2=0x7f37240ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 -- 172.21.15.43:0/709018385 shutdown_connections 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.025+0000 7f372c3a0640 1 -- 172.21.15.43:0/709018385 wait complete. 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 Processor -- start 2024-09-13T13:47:53.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 -- start start 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724101bf0 0x7f37241987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 0x7f3724198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3724107f70 0x7f372419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3724111100 con 0x7f3724101bf0 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3724110f80 con 0x7f3724107f70 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372c3a0640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 -- 0x7f3724111280 con 0x7f37241075f0 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724101bf0 0x7f37241987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372993b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 0x7f3724198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724101bf0 0x7f37241987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41152/0 (socket says 172.21.15.43:41152) 2024-09-13T13:47:53.030 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372993b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 0x7f3724198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:51624/0 (socket says 172.21.15.43:51624) 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 -- 172.21.15.43:0/3043538229 learned_addr learned my addr 172.21.15.43:0/3043538229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a93d640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3724107f70 0x7f372419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 -- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 msgr2=0x7f3724198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 0x7f3724198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 -- 172.21.15.43:0/3043538229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3724107f70 msgr2=0x7f372419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3724107f70 0x7f372419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a13c640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372419e880 con 0x7f3724101bf0 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372993b640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 0x7f3724198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.026+0000 7f372a93d640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3724107f70 0x7f372419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.027+0000 7f372a13c640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724101bf0 0x7f37241987e0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f372000b4a0 tx=0x7f372000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.027+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720014070 con 0x7f3724101bf0 2024-09-13T13:47:53.031 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.027+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3720010a60 con 0x7f3724101bf0 2024-09-13T13:47:53.032 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.027+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372419eb70 con 0x7f3724101bf0 2024-09-13T13:47:53.032 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.027+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200179e0 con 0x7f3724101bf0 2024-09-13T13:47:53.032 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.027+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241a52b0 con 0x7f3724101bf0 2024-09-13T13:47:53.032 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.028+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3724101fd0 con 0x7f3724101bf0 2024-09-13T13:47:53.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.028+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 103136+0+0 (secure 0 0 0) 0x7f3720017070 con 0x7f3724101bf0 2024-09-13T13:47:53.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.029+0000 7f371b7fe640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f371007ae10 0x7f371007d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T13:47:53.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.029+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(763..763 src has 1..763) ==== 8963+0+0 (secure 0 0 0) 0x7f372009dd00 con 0x7f3724101bf0 2024-09-13T13:47:53.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.029+0000 7f372993b640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f371007ae10 0x7f371007d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T13:47:53.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.029+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=764}) -- 0x7f3710086700 con 0x7f3724101bf0 2024-09-13T13:47:53.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.029+0000 7f372993b640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f371007ae10 0x7f371007d2d0 secure :-1 s=READY pgs=3933 cs=0 l=1 rev1=1 crypto rx=0x7f37140095f0 tx=0x7f3714005f90 comp rx=0 tx=0).ready entity=mgr.24458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T13:47:53.034 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.034+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3720068950 con 0x7f3724101bf0 2024-09-13T13:47:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:52 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2956339235' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:53.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:52 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:53.221 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.220+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37241a5570 con 0x7f3724101bf0 2024-09-13T13:47:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:52 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2956339235' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:52 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:53.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:52 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2956339235' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:53.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:52 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:53.805 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.804+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true}]=0 pool '8ef39e5e-57a7-4748-b29c-009db0c54857' removed v764) ==== 248+0+0 (secure 0 0 0) 0x7f372006d800 con 0x7f3724101bf0 2024-09-13T13:47:53.806 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.805+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(764..764 src has 1..764) ==== 296+0+0 (secure 0 0 0) 0x7f372009b600 con 0x7f3724101bf0 2024-09-13T13:47:53.806 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.805+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=765}) -- 0x7f37100873e0 con 0x7f3724101bf0 2024-09-13T13:47:53.928 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.926+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37241a57f0 con 0x7f3724101bf0 2024-09-13T13:47:53.928 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.927+0000 7f371b7fe640 1 -- 172.21.15.43:0/3043538229 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true}]=0 pool '8ef39e5e-57a7-4748-b29c-009db0c54857' does not exist v764) ==== 255+0+0 (secure 0 0 0) 0x7f3720060190 con 0x7f3724101bf0 2024-09-13T13:47:53.928 INFO:tasks.workunit.client.0.smithi043.stderr:pool '8ef39e5e-57a7-4748-b29c-009db0c54857' does not exist 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f371007ae10 msgr2=0x7f371007d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f371007ae10 0x7f371007d2d0 secure :-1 s=READY pgs=3933 cs=0 l=1 rev1=1 crypto rx=0x7f37140095f0 tx=0x7f3714005f90 comp rx=0 tx=0).stop 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724101bf0 msgr2=0x7f37241987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724101bf0 0x7f37241987e0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f372000b4a0 tx=0x7f372000b960 comp rx=0 tx=0).stop 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 shutdown_connections 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3724107f70 0x7f372419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:6800/3978596566,v1:172.21.15.43:6801/3978596566] conn(0x7f371007ae10 0x7f371007d2d0 unknown :-1 s=CLOSED pgs=3933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37241075f0 0x7f3724198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 --2- 172.21.15.43:0/3043538229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3724101bf0 0x7f37241987e0 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 >> 172.21.15.43:0/3043538229 conn(0x7f37240fd630 msgr2=0x7f372406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 shutdown_connections 2024-09-13T13:47:53.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-09-13T13:47:53.929+0000 7f372c3a0640 1 -- 172.21.15.43:0/3043538229 wait complete. 2024-09-13T13:47:53.940 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo OK 2024-09-13T13:47:53.941 INFO:tasks.workunit.client.0.smithi043.stdout:OK 2024-09-13T13:47:53.942 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T13:47:53.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T13:47:54.009 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-13T13:47:54.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-13T13:47:54.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[30106]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[30106]: osdmap e763: 8 total, 8 up, 8 in 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/2956339235' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[30106]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[40255]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:47:54.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[40255]: osdmap e763: 8 total, 8 up, 8 in 2024-09-13T13:47:54.074 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/2956339235' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.074 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[40255]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.074 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:53 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:53 smithi189 ceph-mon[32912]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-13T13:47:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:47:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:53 smithi189 ceph-mon[32912]: osdmap e763: 8 total, 8 up, 8 in 2024-09-13T13:47:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/2956339235' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:53 smithi189 ceph-mon[32912]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "pool2": "002fcfa4-dfdd-44b5-b965-f404930ab85a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:53 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.354 DEBUG:teuthology.parallel:result is None 2024-09-13T13:47:54.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T13:47:54.420 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T13:47:54.420 DEBUG:teuthology.orchestra.run.smithi043:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-13T13:47:54.487 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T13:47:54.487 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-13T13:47:54.499 INFO:tasks.cephadm:Teardown begin 2024-09-13T13:47:54.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T13:47:54.553 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T13:47:54.585 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-13T13:47:54.585 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T13:47:54.609 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T13:47:54.644 INFO:tasks.cephadm:Stopping all daemons... 2024-09-13T13:47:54.644 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-13T13:47:54.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a 2024-09-13T13:47:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:54 smithi189 ceph-mon[32912]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:47:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:47:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:54 smithi189 ceph-mon[32912]: osdmap e764: 8 total, 8 up, 8 in 2024-09-13T13:47:54.898 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:54 smithi189 ceph-mon[32912]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[40255]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:47:54.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:47:54.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[40255]: osdmap e764: 8 total, 8 up, 8 in 2024-09-13T13:47:54.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[40255]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:54.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:54 smithi043 systemd[1]: Stopping Ceph mon.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:47:54.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[30106]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-13T13:47:54.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T13:47:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[30106]: osdmap e764: 8 total, 8 up, 8 in 2024-09-13T13:47:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-mon[30106]: from='client.? 172.21.15.43:0/3043538229' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "8ef39e5e-57a7-4748-b29c-009db0c54857", "pool2": "8ef39e5e-57a7-4748-b29c-009db0c54857", "yes_i_really_really_mean_it": true} : dispatch 2024-09-13T13:47:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a[30083]: 2024-09-13T13:47:54.988+0000 7f077bb6d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:47:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:54 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a[30083]: 2024-09-13T13:47:54.988+0000 7f077bb6d640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-13T13:47:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:55 smithi043 podman[147378]: 2024-09-13 13:47:55.050512941 +0000 UTC m=+0.206915835 container died 329a7351ac3999bf26927d04e6805adda411884207f7af6df9151179965f00c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-13T13:47:55.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:55 smithi043 podman[147378]: 2024-09-13 13:47:55.478914241 +0000 UTC m=+0.635317124 container cleanup 329a7351ac3999bf26927d04e6805adda411884207f7af6df9151179965f00c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, maintainer=Guillaume Abrioux ) 2024-09-13T13:47:55.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:55 smithi043 bash[147378]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a 2024-09-13T13:47:55.924 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:55 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: ::ffff:172.21.15.189 - - [13/Sep/2024:13:47:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-13T13:47:55.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:55 smithi043 podman[147396]: 2024-09-13 13:47:55.67247057 +0000 UTC m=+0.617545534 container remove 329a7351ac3999bf26927d04e6805adda411884207f7af6df9151179965f00c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-13T13:47:55.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:55 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-13T13:47:55.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:55 smithi043 systemd[1]: Stopped Ceph mon.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:47:55.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 13 13:47:55 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service: Consumed 38.677s CPU time. 2024-09-13T13:47:55.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.a.service' 2024-09-13T13:47:55.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:47:55.964 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-13T13:47:55.964 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-13T13:47:55.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.c 2024-09-13T13:47:56.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:56 smithi043 systemd[1]: Stopping Ceph mon.c for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:47:56.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:47:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug there is no tcmu-runner data available 2024-09-13T13:47:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c[40232]: 2024-09-13T13:47:56.319+0000 7ff61766b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:47:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c[40232]: 2024-09-13T13:47:56.319+0000 7ff61766b640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-13T13:47:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:56 smithi043 podman[147512]: 2024-09-13 13:47:56.371270299 +0000 UTC m=+0.174607121 container died 86b07c1f07737f0e303a362e0ac36ff224d298c79176726d6ec711476a2db900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:47:56.619 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:47:56] ENGINE Bus STOPPING 2024-09-13T13:47:56.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:56 smithi043 podman[147512]: 2024-09-13 13:47:56.709571479 +0000 UTC m=+0.512908290 container cleanup 86b07c1f07737f0e303a362e0ac36ff224d298c79176726d6ec711476a2db900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8) 2024-09-13T13:47:56.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 13 13:47:56 smithi043 bash[147512]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-c 2024-09-13T13:47:56.921 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:47:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-13T13:47:56.921 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:47:56] ENGINE Bus STOPPED 2024-09-13T13:47:56.921 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:47:56] ENGINE Bus STARTING 2024-09-13T13:47:56.922 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:47:56] ENGINE Serving on http://:::9283 2024-09-13T13:47:56.922 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:56 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: [13/Sep/2024:13:47:56] ENGINE Bus STARTED 2024-09-13T13:47:57.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.c.service' 2024-09-13T13:47:57.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:47:57.180 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-13T13:47:57.180 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-13T13:47:57.180 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.b 2024-09-13T13:47:57.486 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:57 smithi189 systemd[1]: Stopping Ceph mon.b for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:47:57.810 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:57 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-b[32880]: 2024-09-13T13:47:57.484+0000 7f7cd8e55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:47:57.810 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:57 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-b[32880]: 2024-09-13T13:47:57.484+0000 7f7cd8e55640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-13T13:47:57.810 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:57 smithi189 podman[67085]: 2024-09-13 13:47:57.526937646 +0000 UTC m=+0.148327580 container died 37b7230bc7f5c2f8f8927836ac88199f73a0bb48ce95a28246b127e6b4a9b99b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, ceph=True) 2024-09-13T13:47:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:58 smithi189 podman[67085]: 2024-09-13 13:47:58.025521235 +0000 UTC m=+0.646911167 container cleanup 37b7230bc7f5c2f8f8927836ac88199f73a0bb48ce95a28246b127e6b4a9b99b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-b, ceph=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-13T13:47:58.148 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 13 13:47:58 smithi189 bash[67085]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mon-b 2024-09-13T13:47:58.446 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mon.b.service' 2024-09-13T13:47:58.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:47:58.502 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-13T13:47:58.502 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-13T13:47:58.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.y 2024-09-13T13:47:58.810 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:58 smithi043 systemd[1]: Stopping Ceph mgr.y for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:47:59.085 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:58 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:47:58.808+0000 7fc98898b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:47:59.085 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:58 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y[30358]: 2024-09-13T13:47:58.808+0000 7fc98898b640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-13T13:47:59.085 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:58 smithi043 podman[147654]: 2024-09-13 13:47:58.841188409 +0000 UTC m=+0.141287599 container died d9476f937c897248f0f5f157aa9a5f89c561acb501198c8387deb7577b08b8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T13:47:59.393 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:59 smithi043 podman[147654]: 2024-09-13 13:47:59.183460042 +0000 UTC m=+0.483559230 container cleanup d9476f937c897248f0f5f157aa9a5f89c561acb501198c8387deb7577b08b8ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0201b8c79733293453e7f10a10c7fa43119222b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T13:47:59.394 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 13 13:47:59 smithi043 bash[147654]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-y 2024-09-13T13:47:59.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-13T13:47:59.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:47:59.652 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-13T13:47:59.652 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-13T13:47:59.652 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.x 2024-09-13T13:47:59.972 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:47:59 smithi189 systemd[1]: Stopping Ceph mgr.x for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:00.239 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:47:59 smithi189 podman[67219]: 2024-09-13 13:47:59.97209521 +0000 UTC m=+0.136355544 container died 6ff729135afe75df9efee91ef0a142a7bb530652632ba4dca023dcd8406bd378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T13:48:00.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:48:00 smithi189 podman[67219]: 2024-09-13 13:48:00.329583458 +0000 UTC m=+0.493843794 container cleanup 6ff729135afe75df9efee91ef0a142a7bb530652632ba4dca023dcd8406bd378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-13T13:48:00.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 13 13:48:00 smithi189 bash[67219]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-mgr-x 2024-09-13T13:48:00.903 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-13T13:48:00.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:00.938 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-13T13:48:00.938 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-13T13:48:00.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.0 2024-09-13T13:48:01.240 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:00 smithi043 systemd[1]: Stopping Ceph osd.0 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:01.491 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:01 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0[43902]: 2024-09-13T13:48:01.238+0000 7fddbbaaa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:01.492 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:01 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0[43902]: 2024-09-13T13:48:01.238+0000 7fddbbaaa640 -1 osd.0 764 *** Got signal Terminated *** 2024-09-13T13:48:01.492 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:01 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0[43902]: 2024-09-13T13:48:01.238+0000 7fddbbaaa640 -1 osd.0 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:06.848 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:06 smithi043 podman[147787]: 2024-09-13 13:48:06.587369042 +0000 UTC m=+5.454288680 container died 4d2d5bb28584a9c68912ea0f6436f0f32b3aaf1e084d646fa0e099a271a1c9dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T13:48:07.239 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:06 smithi043 podman[147787]: 2024-09-13 13:48:06.945623419 +0000 UTC m=+5.812543046 container cleanup 4d2d5bb28584a9c68912ea0f6436f0f32b3aaf1e084d646fa0e099a271a1c9dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-13T13:48:07.239 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:06 smithi043 bash[147787]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0 2024-09-13T13:48:07.491 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:07 smithi043 podman[147810]: 2024-09-13 13:48:07.238561325 +0000 UTC m=+0.649079008 container remove 4d2d5bb28584a9c68912ea0f6436f0f32b3aaf1e084d646fa0e099a271a1c9dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-13T13:48:07.831 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:07 smithi043 podman[147904]: 2024-09-13 13:48:07.482408093 +0000 UTC m=+0.016667926 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:48:07.831 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:07 smithi043 podman[147904]: 2024-09-13 13:48:07.598577986 +0000 UTC m=+0.132837812 container create 47d403f978f18d1a10e0cdf91a1f1845ceed789a6f50bae52fb15e46cf85a51d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T13:48:07.831 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:07 smithi043 podman[147904]: 2024-09-13 13:48:07.825703884 +0000 UTC m=+0.359963709 container init 47d403f978f18d1a10e0cdf91a1f1845ceed789a6f50bae52fb15e46cf85a51d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T13:48:07.831 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:07 smithi043 podman[147904]: 2024-09-13 13:48:07.830519484 +0000 UTC m=+0.364779319 container start 47d403f978f18d1a10e0cdf91a1f1845ceed789a6f50bae52fb15e46cf85a51d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T13:48:08.241 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:08 smithi043 podman[147904]: 2024-09-13 13:48:08.05953656 +0000 UTC m=+0.593796402 container attach 47d403f978f18d1a10e0cdf91a1f1845ceed789a6f50bae52fb15e46cf85a51d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T13:48:08.242 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 13 13:48:08 smithi043 podman[147904]: 2024-09-13 13:48:08.059874512 +0000 UTC m=+0.594134340 container died 47d403f978f18d1a10e0cdf91a1f1845ceed789a6f50bae52fb15e46cf85a51d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-13T13:48:08.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.0.service' 2024-09-13T13:48:08.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:08.689 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-13T13:48:08.689 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-13T13:48:08.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.1 2024-09-13T13:48:08.992 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:08 smithi043 systemd[1]: Stopping Ceph osd.1 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:09.355 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:09 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1[49825]: 2024-09-13T13:48:09.053+0000 7f4729044640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:09.355 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:09 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1[49825]: 2024-09-13T13:48:09.053+0000 7f4729044640 -1 osd.1 764 *** Got signal Terminated *** 2024-09-13T13:48:09.355 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:09 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1[49825]: 2024-09-13T13:48:09.053+0000 7f4729044640 -1 osd.1 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:14.742 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:14 smithi043 podman[148042]: 2024-09-13 13:48:14.487824236 +0000 UTC m=+5.552160392 container died ae3c4a67c3c080f0954547e3f9d0708cc2abfb04627f594bcbe66789cd80cdb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-13T13:48:15.201 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:15 smithi043 podman[148042]: 2024-09-13 13:48:15.007640385 +0000 UTC m=+6.071976541 container cleanup ae3c4a67c3c080f0954547e3f9d0708cc2abfb04627f594bcbe66789cd80cdb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T13:48:15.201 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:15 smithi043 bash[148042]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1 2024-09-13T13:48:15.491 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:15 smithi043 podman[148068]: 2024-09-13 13:48:15.201069754 +0000 UTC m=+0.709175651 container remove ae3c4a67c3c080f0954547e3f9d0708cc2abfb04627f594bcbe66789cd80cdb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:48:15.896 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:15 smithi043 podman[148157]: 2024-09-13 13:48:15.540876632 +0000 UTC m=+0.017036698 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:48:15.897 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:15 smithi043 podman[148157]: 2024-09-13 13:48:15.760034937 +0000 UTC m=+0.236195046 container create df350e22335da9a3f901f4b65db4f8f3b25cca66469059ef406222caf0621597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-13T13:48:16.206 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:16 smithi043 podman[148157]: 2024-09-13 13:48:16.004148187 +0000 UTC m=+0.480308251 container init df350e22335da9a3f901f4b65db4f8f3b25cca66469059ef406222caf0621597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-13T13:48:16.207 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:16 smithi043 podman[148157]: 2024-09-13 13:48:16.009876254 +0000 UTC m=+0.486036318 container start df350e22335da9a3f901f4b65db4f8f3b25cca66469059ef406222caf0621597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8) 2024-09-13T13:48:16.207 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:16 smithi043 podman[148157]: 2024-09-13 13:48:16.087571724 +0000 UTC m=+0.563731798 container attach df350e22335da9a3f901f4b65db4f8f3b25cca66469059ef406222caf0621597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-13T13:48:16.491 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 13 13:48:16 smithi043 podman[148157]: 2024-09-13 13:48:16.205882284 +0000 UTC m=+0.682042356 container died df350e22335da9a3f901f4b65db4f8f3b25cca66469059ef406222caf0621597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-13T13:48:16.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.1.service' 2024-09-13T13:48:16.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:16.902 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-13T13:48:16.902 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-13T13:48:16.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.2 2024-09-13T13:48:17.239 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:16 smithi043 systemd[1]: Stopping Ceph osd.2 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:17.491 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:17 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2[56012]: 2024-09-13T13:48:17.237+0000 7f6a2e931640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:17.491 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:17 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2[56012]: 2024-09-13T13:48:17.237+0000 7f6a2e931640 -1 osd.2 764 *** Got signal Terminated *** 2024-09-13T13:48:17.491 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:17 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2[56012]: 2024-09-13T13:48:17.237+0000 7f6a2e931640 -1 osd.2 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:23.138 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:22 smithi043 podman[148284]: 2024-09-13 13:48:22.778170564 +0000 UTC m=+5.647539390 container died 9b60d3711eee06915b4fcd8f11c4103d35c2cacfeca81a7027dcea8e2ea0af7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:48:23.491 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:23 smithi043 podman[148284]: 2024-09-13 13:48:23.210324369 +0000 UTC m=+6.079693186 container cleanup 9b60d3711eee06915b4fcd8f11c4103d35c2cacfeca81a7027dcea8e2ea0af7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-13T13:48:23.492 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:23 smithi043 bash[148284]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2 2024-09-13T13:48:23.492 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:23 smithi043 podman[148304]: 2024-09-13 13:48:23.386566122 +0000 UTC m=+0.602826148 container remove 9b60d3711eee06915b4fcd8f11c4103d35c2cacfeca81a7027dcea8e2ea0af7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-13T13:48:24.173 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:23 smithi043 podman[148401]: 2024-09-13 13:48:23.741531177 +0000 UTC m=+0.016426655 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:48:24.174 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:23 smithi043 podman[148401]: 2024-09-13 13:48:23.854124838 +0000 UTC m=+0.129020305 container create bd45455c07601887e40fa181b951a9d0893c236af99d749740240003f3e066eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-13T13:48:24.174 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:24 smithi043 podman[148401]: 2024-09-13 13:48:24.173119844 +0000 UTC m=+0.448015313 container init bd45455c07601887e40fa181b951a9d0893c236af99d749740240003f3e066eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-13T13:48:24.492 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:24 smithi043 podman[148401]: 2024-09-13 13:48:24.179250152 +0000 UTC m=+0.454145622 container start bd45455c07601887e40fa181b951a9d0893c236af99d749740240003f3e066eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-13T13:48:24.492 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:24 smithi043 podman[148401]: 2024-09-13 13:48:24.264502919 +0000 UTC m=+0.539398426 container attach bd45455c07601887e40fa181b951a9d0893c236af99d749740240003f3e066eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8) 2024-09-13T13:48:24.492 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 13 13:48:24 smithi043 podman[148401]: 2024-09-13 13:48:24.37653164 +0000 UTC m=+0.651427115 container died bd45455c07601887e40fa181b951a9d0893c236af99d749740240003f3e066eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:48:25.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.2.service' 2024-09-13T13:48:25.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:25.062 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-13T13:48:25.062 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-13T13:48:25.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.3 2024-09-13T13:48:25.390 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:25 smithi043 systemd[1]: Stopping Ceph osd.3 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:25.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:48:25.389+0000 7f1dad8bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:25.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:48:25.389+0000 7f1dad8bd640 -1 osd.3 764 *** Got signal Terminated *** 2024-09-13T13:48:25.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:25 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:48:25.389+0000 7f1dad8bd640 -1 osd.3 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:28.319 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:27 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:27.979+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:28.320 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:28 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:28.217+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:28.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:28 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:28.504+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:29.008 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:28 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:28.718+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:29.241 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:28 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:48:28.778+0000 7f1da9ed2640 -1 osd.3 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:05.744939+0000 front 2024-09-13T13:48:05.744867+0000 (oldest deadline 2024-09-13T13:48:28.044857+0000) 2024-09-13T13:48:29.335 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:29 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:29.224+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:29.336 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:29 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:29.007+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:29.593 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:29 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:29.459+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:29.898 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:29 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:29.738+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:30.241 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:29 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3[62358]: 2024-09-13T13:48:29.783+0000 7f1da9ed2640 -1 osd.3 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:05.744939+0000 front 2024-09-13T13:48:05.744867+0000 (oldest deadline 2024-09-13T13:48:28.044857+0000) 2024-09-13T13:48:30.263 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:29 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:29.984+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:30.562 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:30 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:30.426+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:30.563 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:30 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:30.262+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:30.898 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:30 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:30.769+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:31.197 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:30 smithi043 podman[148527]: 2024-09-13 13:48:30.941861304 +0000 UTC m=+5.645425154 container died 0723405f88a7a5493c6979d92dbfb8b45afc5bed1dcaec0d82a74ae4f8579967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-13T13:48:31.250 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:30 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:30.963+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:31.491 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:31 smithi043 podman[148527]: 2024-09-13 13:48:31.311786426 +0000 UTC m=+6.015350275 container cleanup 0723405f88a7a5493c6979d92dbfb8b45afc5bed1dcaec0d82a74ae4f8579967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-13T13:48:31.491 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:31 smithi043 bash[148527]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3 2024-09-13T13:48:31.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:31 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:31.439+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:31.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:31 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:31.249+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:31.893 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:31 smithi043 podman[148548]: 2024-09-13 13:48:31.571261845 +0000 UTC m=+0.626208528 container remove 0723405f88a7a5493c6979d92dbfb8b45afc5bed1dcaec0d82a74ae4f8579967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, maintainer=Guillaume Abrioux ) 2024-09-13T13:48:32.148 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:31 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:31.957+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:32.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:31 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:31.784+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:32.160 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:31 smithi043 podman[148643]: 2024-09-13 13:48:31.796627811 +0000 UTC m=+0.016670943 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:48:32.160 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:31 smithi043 podman[148643]: 2024-09-13 13:48:31.897451418 +0000 UTC m=+0.117494542 container create 7bb3c0c96a7835b8f1e96c69fc68d880f9dc09ecb1ef17f9ae296e17db328a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-13T13:48:32.160 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:32 smithi043 podman[148643]: 2024-09-13 13:48:32.076888579 +0000 UTC m=+0.296931708 container init 7bb3c0c96a7835b8f1e96c69fc68d880f9dc09ecb1ef17f9ae296e17db328a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:48:32.160 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:32 smithi043 podman[148643]: 2024-09-13 13:48:32.082177379 +0000 UTC m=+0.302220501 container start 7bb3c0c96a7835b8f1e96c69fc68d880f9dc09ecb1ef17f9ae296e17db328a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T13:48:32.160 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:32 smithi043 podman[148643]: 2024-09-13 13:48:32.160038317 +0000 UTC m=+0.380081453 container attach 7bb3c0c96a7835b8f1e96c69fc68d880f9dc09ecb1ef17f9ae296e17db328a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-13T13:48:32.491 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:32 smithi043 podman[148643]: 2024-09-13 13:48:32.27280075 +0000 UTC m=+0.492843875 container died 7bb3c0c96a7835b8f1e96c69fc68d880f9dc09ecb1ef17f9ae296e17db328a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-13T13:48:32.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:32.458+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:32.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:32.273+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:32.835 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 13 13:48:32 smithi043 podman[148685]: 2024-09-13 13:48:32.813562073 +0000 UTC m=+0.534913760 container remove 7bb3c0c96a7835b8f1e96c69fc68d880f9dc09ecb1ef17f9ae296e17db328a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-13T13:48:32.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.3.service' 2024-09-13T13:48:32.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:32.884 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-13T13:48:32.884 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-13T13:48:32.884 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.4 2024-09-13T13:48:33.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:32.784+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:33.148 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:32 smithi189 systemd[1]: Stopping Ceph osd.4 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:33.149 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:32 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:32.993+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:33.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:33.484+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:33.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:33.271+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:33.649 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:33.274+0000 7f6dc4eba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:33.649 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:33.274+0000 7f6dc4eba640 -1 osd.4 764 *** Got signal Terminated *** 2024-09-13T13:48:33.649 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:33.274+0000 7f6dc4eba640 -1 osd.4 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:34.011 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:33.749+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:34.011 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:48:33.911Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-13T13:48:34.011 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:48:33.913Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-13T13:48:34.012 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:48:33.914Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-13T13:48:34.012 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:48:33.915Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-13T13:48:34.012 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:48:33.915Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-13T13:48:34.012 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:48:33.915Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-13T13:48:34.012 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:48:33 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:48:33.919Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-13T13:48:34.269 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:34 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:34.010+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:34.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:34 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:34.268+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:34.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:34 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:34.478+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:35.017 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:34 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:34.739+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:35.398 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:35.017+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:35.398 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:35.017+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:11.461765+0000 front 2024-09-13T13:48:11.461777+0000 (oldest deadline 2024-09-13T13:48:34.961701+0000) 2024-09-13T13:48:35.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:35.243+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:35.399 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:35.243+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:35.736 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:35.445+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:35.736 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:35.445+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:36.043 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:35.736+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:36.044 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:35.736+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:36.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:36.277+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:36.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:36.277+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:36.398 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:36.042+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:36.398 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:36.042+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:11.461765+0000 front 2024-09-13T13:48:11.461777+0000 (oldest deadline 2024-09-13T13:48:34.961701+0000) 2024-09-13T13:48:36.718 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:36.408+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:36.718 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:36.408+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:37.039 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:36.717+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:37.039 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:36 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:36.717+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:37.396 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:37.037+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:37.396 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:37.037+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:11.461765+0000 front 2024-09-13T13:48:11.461777+0000 (oldest deadline 2024-09-13T13:48:34.961701+0000) 2024-09-13T13:48:37.396 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:37.253+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:37.396 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:37.253+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:37.491 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:37 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:37.017Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:48:37.491 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:37 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:37.017Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-13T13:48:37.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:37.395+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:37.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:37.395+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:37.995 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:37.729+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:37.995 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:37.729+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:38.368 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:37.994+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:02.061183+0000 front 2024-09-13T13:48:02.061053+0000 (oldest deadline 2024-09-13T13:48:27.960934+0000) 2024-09-13T13:48:38.368 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:37 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4[37513]: 2024-09-13T13:48:37.994+0000 7f6dc14cf640 -1 osd.4 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:11.461765+0000 front 2024-09-13T13:48:11.461777+0000 (oldest deadline 2024-09-13T13:48:34.961701+0000) 2024-09-13T13:48:38.368 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:38 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:38.230+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:38.368 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:38 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:38.230+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:38.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:38 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:38.366+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:38.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:38 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:38.366+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:39.108 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:38 smithi189 podman[67364]: 2024-09-13 13:48:38.809371984 +0000 UTC m=+5.728178967 container died 8114337b1f26a7c2aa5a24a4cbf4dfbc09fd5016a64eaaf74e79d39630661818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-13T13:48:39.109 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:38 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:38.721+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:39.109 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:38 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:38.721+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:39.398 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:39 smithi189 podman[67364]: 2024-09-13 13:48:39.20608825 +0000 UTC m=+6.124895235 container cleanup 8114337b1f26a7c2aa5a24a4cbf4dfbc09fd5016a64eaaf74e79d39630661818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-13T13:48:39.398 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:39 smithi189 bash[67364]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4 2024-09-13T13:48:39.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:39 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:39.257+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:39.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:39 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:39.257+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:39.696 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:39 smithi189 podman[67387]: 2024-09-13 13:48:39.436815743 +0000 UTC m=+0.626661461 container remove 8114337b1f26a7c2aa5a24a4cbf4dfbc09fd5016a64eaaf74e79d39630661818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8) 2024-09-13T13:48:39.696 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:39 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:39.409+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:39.696 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:39 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:39.409+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:40.019 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:39 smithi189 podman[67482]: 2024-09-13 13:48:39.673471102 +0000 UTC m=+0.015914570 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:48:40.020 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:39 smithi189 podman[67482]: 2024-09-13 13:48:39.8084249 +0000 UTC m=+0.150868397 container create ff37095c86dbaf491e4b6892cf56638d0b902afaf7fa3cc1a783019b0a956162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-13T13:48:40.020 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:39 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:39.695+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:40.020 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:39 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:39.695+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:40.393 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:40 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:40.257+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:40.394 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:40 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:40.257+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:40.394 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:40 smithi189 podman[67482]: 2024-09-13 13:48:40.018857604 +0000 UTC m=+0.361301062 container init ff37095c86dbaf491e4b6892cf56638d0b902afaf7fa3cc1a783019b0a956162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-13T13:48:40.394 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:40 smithi189 podman[67482]: 2024-09-13 13:48:40.024338511 +0000 UTC m=+0.366781971 container start ff37095c86dbaf491e4b6892cf56638d0b902afaf7fa3cc1a783019b0a956162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-13T13:48:40.394 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:40 smithi189 podman[67482]: 2024-09-13 13:48:40.102098515 +0000 UTC m=+0.444541969 container attach ff37095c86dbaf491e4b6892cf56638d0b902afaf7fa3cc1a783019b0a956162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:48:40.394 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 13 13:48:40 smithi189 podman[67482]: 2024-09-13 13:48:40.217492453 +0000 UTC m=+0.559935921 container died ff37095c86dbaf491e4b6892cf56638d0b902afaf7fa3cc1a783019b0a956162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T13:48:40.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:40 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:40.393+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:40.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:40 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:40.393+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:40.826 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.4.service' 2024-09-13T13:48:40.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:40.866 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-13T13:48:40.866 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-13T13:48:40.866 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.5 2024-09-13T13:48:40.926 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:40 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:40.650+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:40.926 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:40 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:40.650+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:41.186 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:40 smithi189 systemd[1]: Stopping Ceph osd.5 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:41.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:41.297+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:41.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:41.297+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:41.649 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:41.184+0000 7fea0508f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:41.649 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:41.184+0000 7fea0508f640 -1 osd.5 764 *** Got signal Terminated *** 2024-09-13T13:48:41.649 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:41.184+0000 7fea0508f640 -1 osd.5 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:41.649 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:41.394+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:41.649 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:41.394+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:42.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:41.698+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:42.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:41 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:41.698+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:42.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:42 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:42.297+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:42.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:42 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:42.297+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:42.649 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:42 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:42.348+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:42.649 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:42 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:42.348+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:43.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:42 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:42.699+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:43.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:42 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:42.699+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:43.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:43 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:43.359+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:43.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:43 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:43.359+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:43.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:43 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:43.284+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:43.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:43 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:43.284+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:44.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:43 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:43.710+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:44.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:43 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:43.710+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:44.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:43 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:43.710+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:44.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:44.315+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:44.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:44.315+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:44.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:44.320+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:44.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:44.320+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:44.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:44.320+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:21.383575+0000 front 2024-09-13T13:48:21.383567+0000 (oldest deadline 2024-09-13T13:48:43.583861+0000) 2024-09-13T13:48:45.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:44.672+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:45.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:44.672+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:45.149 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:44 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:44.672+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:45.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:45.315+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:04.481371+0000 front 2024-09-13T13:48:04.481533+0000 (oldest deadline 2024-09-13T13:48:27.981414+0000) 2024-09-13T13:48:45.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:45.315+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.282489+0000 front 2024-09-13T13:48:10.282416+0000 (oldest deadline 2024-09-13T13:48:34.982285+0000) 2024-09-13T13:48:45.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5[43304]: 2024-09-13T13:48:45.315+0000 7fe9fd69c640 -1 osd.5 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:21.383575+0000 front 2024-09-13T13:48:21.383567+0000 (oldest deadline 2024-09-13T13:48:43.583861+0000) 2024-09-13T13:48:45.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:45.332+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:45.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:45.332+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:46.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:45.681+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:46.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:45.681+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:46.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:45 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:45.681+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:46.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:46.372+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:46.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:46.372+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:46.967 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:46 smithi189 podman[67611]: 2024-09-13 13:48:46.671438168 +0000 UTC m=+5.587265457 container died c7fab5d554fa6f2d421379f083725e882ad57c5c393ae3d549634362cd858a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-13T13:48:46.968 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:46.703+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:46.968 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:46.703+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:46.968 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:46 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:46.703+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:47.349 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 podman[67611]: 2024-09-13 13:48:47.066136167 +0000 UTC m=+5.981963457 container cleanup c7fab5d554fa6f2d421379f083725e882ad57c5c393ae3d549634362cd858a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-13T13:48:47.349 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 bash[67611]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5 2024-09-13T13:48:47.492 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:47 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:47.017Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-13T13:48:47.492 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:47 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:47.018Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:48:47.492 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:47 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:47.018Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-13T13:48:47.648 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 podman[67632]: 2024-09-13 13:48:47.350576327 +0000 UTC m=+0.675261553 container remove c7fab5d554fa6f2d421379f083725e882ad57c5c393ae3d549634362cd858a66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-13T13:48:47.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:47 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:47.347+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:47.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:47 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:47.347+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:47.987 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 podman[67722]: 2024-09-13 13:48:47.594505092 +0000 UTC m=+0.015763247 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:48:47.988 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 podman[67722]: 2024-09-13 13:48:47.701466176 +0000 UTC m=+0.122724323 container create 9d8e9112e72982dba2118ef6f7365e93cf2d75f3979e142bfd3ade67ad71373b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:48:47.988 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 podman[67722]: 2024-09-13 13:48:47.904276047 +0000 UTC m=+0.325534196 container init 9d8e9112e72982dba2118ef6f7365e93cf2d75f3979e142bfd3ade67ad71373b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-13T13:48:47.988 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 podman[67722]: 2024-09-13 13:48:47.908862896 +0000 UTC m=+0.330121048 container start 9d8e9112e72982dba2118ef6f7365e93cf2d75f3979e142bfd3ade67ad71373b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-13T13:48:47.988 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:47 smithi189 podman[67722]: 2024-09-13 13:48:47.987082425 +0000 UTC m=+0.408340584 container attach 9d8e9112e72982dba2118ef6f7365e93cf2d75f3979e142bfd3ade67ad71373b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:48:47.988 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:47 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:47.683+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:47.988 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:47 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:47.683+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:47.989 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:47 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:47.683+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:48.348 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 13 13:48:48 smithi189 podman[67722]: 2024-09-13 13:48:48.096945434 +0000 UTC m=+0.518203583 container died 9d8e9112e72982dba2118ef6f7365e93cf2d75f3979e142bfd3ade67ad71373b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-13T13:48:48.640 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:48 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:48.346+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:48.641 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:48 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:48.346+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:48.641 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:48 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:48.346+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:48.664 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.5.service' 2024-09-13T13:48:48.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:48.745 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-13T13:48:48.745 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-13T13:48:48.745 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.6 2024-09-13T13:48:48.898 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:48 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:48.695+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:48.898 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:48 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:48.695+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:48.898 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:48 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:48.695+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:48.898 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:48 smithi189 systemd[1]: Stopping Ceph osd.6 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:49.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:49.312+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:49.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:49.312+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:49.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:49.312+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:49.399 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:49.086+0000 7f8372db2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:49.399 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:49.086+0000 7f8372db2640 -1 osd.6 764 *** Got signal Terminated *** 2024-09-13T13:48:49.399 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:49.086+0000 7f8372db2640 -1 osd.6 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:50.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:49.708+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:50.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:49.708+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:50.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:49 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:49.708+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:50.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:50 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:50.361+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:50.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:50 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:50.361+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:50.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:50 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:50.361+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:51.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:50 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:50.735+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:51.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:50 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:50.735+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:51.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:50 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:50.735+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:51.757 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:51 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:51.409+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:51.757 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:51 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:51.409+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:51.757 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:51 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:51.409+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:52.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:51 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:51.755+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:52.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:51 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:51.755+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:52.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:51 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:51.755+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:52.749 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:52 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:52.441+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:52.749 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:52 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:52.441+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:52.749 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:52 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:52.441+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:52.749 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:52 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:52.441+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:53.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:52 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:52.748+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:53.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:52 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:52.748+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:53.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:52 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:52.748+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:53.764 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:53.482+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:53.774 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:53.482+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:53.774 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:53.482+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:53.774 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:53.482+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:53.774 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:53.482+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:48:54.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:53.763+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.454519+0000 front 2024-09-13T13:48:03.454607+0000 (oldest deadline 2024-09-13T13:48:28.654827+0000) 2024-09-13T13:48:54.149 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:53.763+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:13.656981+0000 front 2024-09-13T13:48:13.656998+0000 (oldest deadline 2024-09-13T13:48:35.356745+0000) 2024-09-13T13:48:54.149 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:53.763+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:18.257689+0000 front 2024-09-13T13:48:18.257740+0000 (oldest deadline 2024-09-13T13:48:43.457862+0000) 2024-09-13T13:48:54.149 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:53 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6[49000]: 2024-09-13T13:48:53.763+0000 7f836f3c7640 -1 osd.6 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.758569+0000 front 2024-09-13T13:48:28.758592+0000 (oldest deadline 2024-09-13T13:48:52.858602+0000) 2024-09-13T13:48:54.703 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:54 smithi189 podman[67850]: 2024-09-13 13:48:54.415806799 +0000 UTC m=+5.439484163 container died 3406c7fc28b64c6906ffc549243081e3b577abc1b21feeb1bfc4f5b226a7663c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-13T13:48:54.704 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:54 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:54.478+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:54.704 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:54 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:54.478+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:54.704 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:54 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:54.478+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:54.704 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:54 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:54.478+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:54.704 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:54 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:54.478+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:48:55.054 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:54 smithi189 podman[67850]: 2024-09-13 13:48:54.825793451 +0000 UTC m=+5.849470812 container cleanup 3406c7fc28b64c6906ffc549243081e3b577abc1b21feeb1bfc4f5b226a7663c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-13T13:48:55.054 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:54 smithi189 bash[67850]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6 2024-09-13T13:48:55.398 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:55 smithi189 podman[67871]: 2024-09-13 13:48:55.054114272 +0000 UTC m=+0.632820782 container remove 3406c7fc28b64c6906ffc549243081e3b577abc1b21feeb1bfc4f5b226a7663c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-13T13:48:55.733 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:55 smithi189 podman[67965]: 2024-09-13 13:48:55.397509532 +0000 UTC m=+0.016134029 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:48:55.733 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:55 smithi189 podman[67965]: 2024-09-13 13:48:55.504713543 +0000 UTC m=+0.123338030 container create c29540e535523b4491ff74ee757c2944fabbc2e173d6c18e2f451672746a6784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-13T13:48:55.733 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:55.466+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:55.733 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:55.466+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:55.734 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:55.466+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:55.734 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:55.466+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:55.734 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:55 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:55.466+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:48:56.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:55 smithi189 podman[67965]: 2024-09-13 13:48:55.732240007 +0000 UTC m=+0.350864495 container init c29540e535523b4491ff74ee757c2944fabbc2e173d6c18e2f451672746a6784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-13T13:48:56.148 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:55 smithi189 podman[67965]: 2024-09-13 13:48:55.737653011 +0000 UTC m=+0.356277500 container start c29540e535523b4491ff74ee757c2944fabbc2e173d6c18e2f451672746a6784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-13T13:48:56.149 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:55 smithi189 podman[67965]: 2024-09-13 13:48:55.834465985 +0000 UTC m=+0.453090488 container attach c29540e535523b4491ff74ee757c2944fabbc2e173d6c18e2f451672746a6784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, maintainer=Guillaume Abrioux ) 2024-09-13T13:48:56.149 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 13 13:48:55 smithi189 podman[67965]: 2024-09-13 13:48:55.926201864 +0000 UTC m=+0.544826362 container died c29540e535523b4491ff74ee757c2944fabbc2e173d6c18e2f451672746a6784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-13T13:48:56.511 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.446+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:56.511 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.446+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:56.511 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.446+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:56.511 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.446+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:56.511 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.446+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:48:56.525 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.6.service' 2024-09-13T13:48:56.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:48:56.565 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-13T13:48:56.565 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-13T13:48:56.565 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.7 2024-09-13T13:48:56.898 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 systemd[1]: Stopping Ceph osd.7 for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:48:57.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.898+0000 7fcafcb68640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:48:57.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.898+0000 7fcafcb68640 -1 osd.7 764 *** Got signal Terminated *** 2024-09-13T13:48:57.398 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:56 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:56.898+0000 7fcafcb68640 -1 osd.7 764 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T13:48:57.491 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:57.018Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:48:57.491 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:57.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:48:57.492 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:48:57 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:48:57.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-13T13:48:57.898 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:57 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:57.416+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:57.898 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:57 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:57.416+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:57.898 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:57 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:57.416+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:57.898 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:57 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:57.416+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:57.898 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:57 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:57.416+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:48:58.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:58 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:58.371+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:58.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:58 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:58.371+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:58.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:58 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:58.371+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:58.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:58 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:58.371+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:58.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:58 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:58.371+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:48:59.664 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:59 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:59.397+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:48:59.664 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:59 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:59.397+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:48:59.665 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:59 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:59.397+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:48:59.665 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:59 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:59.397+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:48:59.665 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:48:59 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:48:59.397+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:49:00.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:00 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:00.361+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:49:00.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:00 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:00.361+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:49:00.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:00 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:00.361+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:49:00.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:00 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:00.361+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:49:00.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:00 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:00.361+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:49:01.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:01 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:01.328+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-13T13:48:03.508278+0000 front 2024-09-13T13:48:03.508330+0000 (oldest deadline 2024-09-13T13:48:28.208117+0000) 2024-09-13T13:49:01.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:01 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:01.328+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-09-13T13:48:10.409503+0000 front 2024-09-13T13:48:10.409575+0000 (oldest deadline 2024-09-13T13:48:34.509428+0000) 2024-09-13T13:49:01.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:01 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:01.328+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-09-13T13:48:16.810453+0000 front 2024-09-13T13:48:16.810434+0000 (oldest deadline 2024-09-13T13:48:48.010713+0000) 2024-09-13T13:49:01.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:01 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:01.328+0000 7fcaf5175640 -1 osd.7 764 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-09-13T13:48:28.011181+0000 front 2024-09-13T13:48:28.011134+0000 (oldest deadline 2024-09-13T13:48:51.511182+0000) 2024-09-13T13:49:01.649 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:01 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7[55093]: 2024-09-13T13:49:01.328+0000 7fcaf5175640 -1 osd.7 764 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14526.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e764) 2024-09-13T13:49:02.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:02 smithi189 podman[68094]: 2024-09-13 13:49:02.381371046 +0000 UTC m=+5.590177033 container died 897a64809a61789ae52f8f54b03e51c949393950139bd7ecd811d5ae459867ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-13T13:49:03.006 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:02 smithi189 podman[68094]: 2024-09-13 13:49:02.754144954 +0000 UTC m=+5.962950950 container cleanup 897a64809a61789ae52f8f54b03e51c949393950139bd7ecd811d5ae459867ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T13:49:03.006 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:02 smithi189 bash[68094]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7 2024-09-13T13:49:03.369 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:03 smithi189 podman[68129]: 2024-09-13 13:49:03.005003737 +0000 UTC m=+0.618057275 container remove 897a64809a61789ae52f8f54b03e51c949393950139bd7ecd811d5ae459867ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-13T13:49:03.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:03 smithi189 podman[68223]: 2024-09-13 13:49:03.271836133 +0000 UTC m=+0.016028640 image pull 1641fe80ce7289a76d112fafe763b98dbdcc395dba398d6702278c8c710a2031 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b 2024-09-13T13:49:03.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:03 smithi189 podman[68223]: 2024-09-13 13:49:03.372331816 +0000 UTC m=+0.116524320 container create 1614782d02d99ac0ffe42499e77af53cf4c45afca77d33ac6ae39053a9ecabb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-13T13:49:03.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:03 smithi189 podman[68223]: 2024-09-13 13:49:03.591569837 +0000 UTC m=+0.335762344 container init 1614782d02d99ac0ffe42499e77af53cf4c45afca77d33ac6ae39053a9ecabb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8) 2024-09-13T13:49:03.648 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:03 smithi189 podman[68223]: 2024-09-13 13:49:03.596338578 +0000 UTC m=+0.340531080 container start 1614782d02d99ac0ffe42499e77af53cf4c45afca77d33ac6ae39053a9ecabb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8) 2024-09-13T13:49:04.078 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:03 smithi189 podman[68223]: 2024-09-13 13:49:03.683032588 +0000 UTC m=+0.427225105 container attach 1614782d02d99ac0ffe42499e77af53cf4c45afca77d33ac6ae39053a9ecabb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-13T13:49:04.079 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 13 13:49:03 smithi189 podman[68223]: 2024-09-13 13:49:03.795213303 +0000 UTC m=+0.539405807 container died 1614782d02d99ac0ffe42499e77af53cf4c45afca77d33ac6ae39053a9ecabb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-13T13:49:04.372 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@osd.7.service' 2024-09-13T13:49:04.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:49:04.411 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-13T13:49:04.411 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-13T13:49:04.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-13T13:49:04.717 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:04 smithi043 systemd[1]: Stopping Ceph rgw.foo.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:49:04.991 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:04 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a[66785]: 2024-09-13T13:49:04.715+0000 7f4f2663d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T13:49:04.991 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:04 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a[66785]: 2024-09-13T13:49:04.715+0000 7f4f29ea9880 -1 shutting down 2024-09-13T13:49:07.492 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:07 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:49:07.018Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:49:07.492 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:07 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:49:07.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:49:07.492 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:07 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:49:07.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-13T13:49:14.992 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:14 smithi043 bash[148781]: time="2024-09-13T13:49:14Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-13T13:49:14.992 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:14 smithi043 podman[148781]: 2024-09-13 13:49:14.750998685 +0000 UTC m=+10.139731253 container died cd94fd90adfe7fb981cc80f2273257a3e0dc4a38f09e171eaed5a9692436746b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-13T13:49:15.459 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:15 smithi043 podman[148781]: 2024-09-13 13:49:15.087379026 +0000 UTC m=+10.476111590 container cleanup cd94fd90adfe7fb981cc80f2273257a3e0dc4a38f09e171eaed5a9692436746b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-13T13:49:15.460 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:15 smithi043 bash[148781]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-13T13:49:15.460 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:15 smithi043 podman[148816]: 2024-09-13 13:49:15.238280063 +0000 UTC m=+0.489534676 container remove cd94fd90adfe7fb981cc80f2273257a3e0dc4a38f09e171eaed5a9692436746b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-13T13:49:15.460 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 13 13:49:15 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-13T13:49:15.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-13T13:49:15.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:49:15.521 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-13T13:49:15.521 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-13T13:49:15.522 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@prometheus.a 2024-09-13T13:49:15.832 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 systemd[1]: Stopping Ceph prometheus.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:49:16.089 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.831Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-13T13:49:16.089 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-13T13:49:16.089 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-13T13:49:16.089 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-13T13:49:16.089 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.832Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.833Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.833Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a[63756]: ts=2024-09-13T13:49:15.833Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-13T13:49:16.090 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:15 smithi189 podman[68357]: 2024-09-13 13:49:15.842878811 +0000 UTC m=+0.119365994 container died 72a37218f40208c469bfef73e1b586e9346d0bde13fc7b45c7d5ded6b7cd32e5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:16.398 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:16 smithi189 podman[68357]: 2024-09-13 13:49:16.195855953 +0000 UTC m=+0.472343097 container cleanup 72a37218f40208c469bfef73e1b586e9346d0bde13fc7b45c7d5ded6b7cd32e5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:16.398 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 13 13:49:16 smithi189 bash[68357]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-prometheus-a 2024-09-13T13:49:16.618 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-13T13:49:16.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T13:49:16.653 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-13T13:49:16.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T13:49:16.866 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:49:17.241 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:17 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:49:17.019Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:49:17.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:17 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:49:17.020Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.43:8443: connect: connection refused" 2024-09-13T13:49:17.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:17 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:49:17.020Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-13T13:49:18.815 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:18 smithi043 systemd[1]: Stopping Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:49:19.110 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:18 smithi043 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a[75635]: ts=2024-09-13T13:49:18.814Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-13T13:49:19.110 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:18 smithi043 podman[149300]: 2024-09-13 13:49:18.86583378 +0000 UTC m=+0.166116628 container died f08c22686d29776f3b0ba541876a82767d2a4b3f588f8559605fe919d78f5569 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:19.394 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:19 smithi043 podman[149300]: 2024-09-13 13:49:19.201374374 +0000 UTC m=+0.501657261 container cleanup f08c22686d29776f3b0ba541876a82767d2a4b3f588f8559605fe919d78f5569 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:19.395 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:19 smithi043 bash[149300]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a 2024-09-13T13:49:19.741 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:19 smithi043 podman[149316]: 2024-09-13 13:49:19.394369925 +0000 UTC m=+0.524499948 container remove f08c22686d29776f3b0ba541876a82767d2a4b3f588f8559605fe919d78f5569 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:19.741 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:19 smithi043 podman[149316]: 2024-09-13 13:49:19.4783726 +0000 UTC m=+0.608502582 volume remove 2d201484b42aabd6a8dac3629ef783d7cd180dca4a67348867ef9ec5a32960f4 2024-09-13T13:49:20.241 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:19 smithi043 systemd[1]: Stopping Ceph node-exporter.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:49:20.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:19 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-13T13:49:20.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:19 smithi043 systemd[1]: Stopped Ceph alertmanager.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:49:20.242 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 13 13:49:19 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.919s CPU time. 2024-09-13T13:49:20.556 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 podman[149436]: 2024-09-13 13:49:20.244162503 +0000 UTC m=+0.124343210 container died 458e3f697bcfe2d59dac647f71c370d835686a43f06eaf541313a440b5a2791b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:20.556 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 podman[149436]: 2024-09-13 13:49:20.555809114 +0000 UTC m=+0.435989822 container cleanup 458e3f697bcfe2d59dac647f71c370d835686a43f06eaf541313a440b5a2791b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:20.967 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 bash[149436]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a 2024-09-13T13:49:20.967 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 podman[149454]: 2024-09-13 13:49:20.740327004 +0000 UTC m=+0.499913370 container remove 458e3f697bcfe2d59dac647f71c370d835686a43f06eaf541313a440b5a2791b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-13T13:49:20.967 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-13T13:49:21.241 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-13T13:49:21.242 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 systemd[1]: Stopped Ceph node-exporter.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:49:21.242 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 13 13:49:20 smithi043 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 3.959s CPU time. 2024-09-13T13:49:21.610 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T13:49:21.812 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:49:23.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:23 smithi189 systemd[1]: Stopping Ceph iscsi.iscsi.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:49:23.399 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:23 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a[59855]: debug Shutdown received 2024-09-13T13:49:33.643 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:33 smithi189 bash[68781]: time="2024-09-13T13:49:33Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-13T13:49:33.643 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:33 smithi189 podman[68781]: 2024-09-13 13:49:33.359732922 +0000 UTC m=+10.127912928 container died 0c06c8b206d63384487c3de7cd29ba89561bdcf61a5e194893fb6fe43ae76d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, org.label-schema.license=GPLv2) 2024-09-13T13:49:33.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:33 smithi189 podman[68781]: 2024-09-13 13:49:33.732330249 +0000 UTC m=+10.500510260 container cleanup 0c06c8b206d63384487c3de7cd29ba89561bdcf61a5e194893fb6fe43ae76d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T13:49:33.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:33 smithi189 bash[68781]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-13T13:49:34.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:33 smithi189 podman[68804]: 2024-09-13 13:49:33.991698549 +0000 UTC m=+0.629010437 container remove 0c06c8b206d63384487c3de7cd29ba89561bdcf61a5e194893fb6fe43ae76d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5ab18e92f956ea4280103e54eef24f76e1a43e77fc4dc78cee3cb12c7a94538b, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-0840-b0201b8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T13:49:34.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:33 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-13T13:49:34.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:34 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-13T13:49:34.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:34 smithi189 systemd[1]: Stopped Ceph iscsi.iscsi.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:49:34.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 13 13:49:34 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.759s CPU time. 2024-09-13T13:49:35.993 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:35 smithi189 systemd[1]: Stopping Ceph grafana.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:49:36.249 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=server t=2024-09-13T13:49:35.991920299Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-13T13:49:36.250 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=ticker t=2024-09-13T13:49:35.992294461Z level=info msg=stopped last_tick=2024-09-13T13:49:30Z 2024-09-13T13:49:36.250 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=tracing t=2024-09-13T13:49:35.992338316Z level=info msg="Closing tracing" 2024-09-13T13:49:36.250 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:35 smithi189 ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a[62469]: logger=grafana-apiserver t=2024-09-13T13:49:35.992358172Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-13T13:49:36.250 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:36 smithi189 podman[69092]: 2024-09-13 13:49:36.002447217 +0000 UTC m=+0.120898670 container died c292aacac233f0ff817ecfce2dd3ea69ef73902fe178d1342cea30291c46dfd6 (image=quay.io/ceph/grafana:10.4.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-13T13:49:36.515 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:36 smithi189 podman[69092]: 2024-09-13 13:49:36.322551987 +0000 UTC m=+0.441003438 container cleanup c292aacac233f0ff817ecfce2dd3ea69ef73902fe178d1342cea30291c46dfd6 (image=quay.io/ceph/grafana:10.4.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-13T13:49:36.516 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:36 smithi189 bash[69092]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a 2024-09-13T13:49:36.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:36 smithi189 podman[69109]: 2024-09-13 13:49:36.515562611 +0000 UTC m=+0.513913929 container remove c292aacac233f0ff817ecfce2dd3ea69ef73902fe178d1342cea30291c46dfd6 (image=quay.io/ceph/grafana:10.4.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-13T13:49:36.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:36 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@grafana.a.service: Deactivated successfully. 2024-09-13T13:49:36.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:36 smithi189 systemd[1]: Stopped Ceph grafana.a for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:49:36.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 13 13:49:36 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@grafana.a.service: Consumed 13.163s CPU time. 2024-09-13T13:49:37.199 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:36 smithi189 systemd[1]: Stopping Ceph node-exporter.b for 39cb2aa4-71d3-11ef-bceb-c7b262605968... 2024-09-13T13:49:37.461 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 podman[69232]: 2024-09-13 13:49:37.198508739 +0000 UTC m=+0.120137222 container died 22f641d70cf75cfbc2503bbc5042f223c78eb14d87633197d7e01493eeb07d8f (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-13T13:49:37.819 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 podman[69232]: 2024-09-13 13:49:37.551324282 +0000 UTC m=+0.472952752 container cleanup 22f641d70cf75cfbc2503bbc5042f223c78eb14d87633197d7e01493eeb07d8f (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-13T13:49:37.819 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 bash[69232]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b 2024-09-13T13:49:38.115 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 podman[69249]: 2024-09-13 13:49:37.819179305 +0000 UTC m=+0.620801024 container remove 22f641d70cf75cfbc2503bbc5042f223c78eb14d87633197d7e01493eeb07d8f (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-13T13:49:38.115 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-13T13:49:38.116 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-13T13:49:38.116 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 systemd[1]: Stopped Ceph node-exporter.b for 39cb2aa4-71d3-11ef-bceb-c7b262605968. 2024-09-13T13:49:38.116 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 13 13:49:37 smithi189 systemd[1]: ceph-39cb2aa4-71d3-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 3.578s CPU time. 2024-09-13T13:49:38.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T13:49:38.675 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T13:49:38.703 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-13T13:49:38.704 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647/remote/smithi043/crash 2024-09-13T13:49:38.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/crash -- . 2024-09-13T13:49:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T13:49:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-09-13T13:49:38.756 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647/remote/smithi189/crash 2024-09-13T13:49:38.757 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/crash -- . 2024-09-13T13:49:38.791 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T13:49:38.791 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-09-13T13:49:38.792 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-13T13:49:38.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v CEPHADM_STRAY_DAEMON | head -n 1 2024-09-13T13:49:38.831 INFO:tasks.cephadm:Compressing logs... 2024-09-13T13:49:38.831 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T13:49:38.874 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T13:49:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T13:49:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-13T13:49:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-13T13:49:38.904 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-13T13:49:38.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-13T13:49:38.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-13T13:49:38.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-13T13:49:38.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-13T13:49:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-13T13:49:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-13T13:49:38.905 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-13T13:49:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.log 2024-09-13T13:49:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/tcmu-runner.log 2024-09-13T13:49:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:: No such file or directory 2024-09-13T13:49:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-13T13:49:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-13T13:49:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/tcmu-runner.log: 63.1% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/tcmu-runner.log.gz/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.7.log: 2024-09-13T13:49:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-13T13:49:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.6.log: 79.9% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.log 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-13T13:49:38.907 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mgr.x.log: 87.1% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-13T13:49:38.909 INFO:teuthology.orchestra.run.smithi189.stderr: 92.5% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-13T13:49:38.912 INFO:teuthology.orchestra.run.smithi189.stderr: 91.6% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-13T13:49:38.913 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-13T13:49:38.913 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 91.5% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-13T13:49:38.915 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.2.log: /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-13T13:49:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.log: 91.4% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-13T13:49:38.919 INFO:teuthology.orchestra.run.smithi043.stderr: 95.3% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-13T13:49:38.921 INFO:teuthology.orchestra.run.smithi043.stderr: 92.8% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-13T13:49:38.928 INFO:teuthology.orchestra.run.smithi189.stderr: 91.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-13T13:49:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-13T13:49:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-13T13:49:38.958 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-13T13:49:38.974 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-13T13:49:39.228 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-13T13:49:39.754 INFO:teuthology.orchestra.run.smithi189.stderr: 92.9% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-13T13:49:40.247 INFO:teuthology.orchestra.run.smithi043.stderr: 92.5% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-13T13:49:41.576 INFO:teuthology.orchestra.run.smithi043.stderr: 92.2% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-13T13:49:44.960 INFO:teuthology.orchestra.run.smithi189.stderr: 93.9% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-13T13:49:45.173 INFO:teuthology.orchestra.run.smithi189.stderr: 93.9% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-13T13:49:45.591 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-13T13:49:45.758 INFO:teuthology.orchestra.run.smithi189.stderr: 94.0% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-13T13:49:45.963 INFO:teuthology.orchestra.run.smithi189.stderr: 93.9% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-13T13:49:45.966 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-13T13:49:45.966 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m7.077s 2024-09-13T13:49:45.966 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m26.192s 2024-09-13T13:49:45.966 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.934s 2024-09-13T13:49:46.222 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-13T13:49:46.259 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-13T13:49:46.596 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/39cb2aa4-71d3-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-13T13:49:46.598 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-13T13:49:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m7.710s 2024-09-13T13:49:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m32.381s 2024-09-13T13:49:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m1.045s 2024-09-13T13:49:46.599 INFO:tasks.cephadm:Archiving logs... 2024-09-13T13:49:46.599 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647/remote/smithi043/log 2024-09-13T13:49:46.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T13:49:50.828 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647/remote/smithi189/log 2024-09-13T13:49:50.829 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T13:49:54.717 INFO:tasks.cephadm:Removing cluster... 2024-09-13T13:49:54.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 --force 2024-09-13T13:49:54.931 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:49:55.229 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 39cb2aa4-71d3-11ef-bceb-c7b262605968 --force 2024-09-13T13:49:55.435 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 39cb2aa4-71d3-11ef-bceb-c7b262605968 2024-09-13T13:49:55.723 INFO:tasks.cephadm:Removing cephadm ... 2024-09-13T13:49:55.723 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T13:49:55.741 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T13:49:55.756 INFO:tasks.cephadm:Teardown complete 2024-09-13T13:49:55.757 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T13:49:55.767 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T13:49:55.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T13:49:55.783 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T13:49:55.888 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T13:49:55.896 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T13:49:55.896 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T13:49:55.899 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T13:49:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-13T13:49:55.913 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-13T13:49:55.917 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T13:49:55.917 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-13T13:49:55.917 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -28us[ -18us] +/- 65ms 2024-09-13T13:49:55.917 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -420us[ -420us] +/- 63ms 2024-09-13T13:49:55.918 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +525us[ +525us] +/- 64ms 2024-09-13T13:49:55.918 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:49:55.919 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T13:49:55.919 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-13T13:49:55.919 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 -57us[ -65us] +/- 65ms 2024-09-13T13:49:55.919 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -430us[ -439us] +/- 63ms 2024-09-13T13:49:55.919 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +498us[ +489us] +/- 64ms 2024-09-13T13:49:55.919 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T13:49:55.920 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T13:49:55.928 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T13:49:55.929 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T13:49:55.937 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T13:49:56.040 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T13:49:56.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T13:49:56.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:49:56.142 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T13:49:56.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:49:56.170 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T13:49:56.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T13:49:56.195 INFO:teuthology.task.internal:Duration was 2015.650959 seconds 2024-09-13T13:49:56.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T13:49:56.204 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T13:49:56.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T13:49:56.207 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T13:49:56.248 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T13:49:56.249 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T13:49:56.538 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T13:49:56.538 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T13:49:56.538 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T13:49:56.602 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-13T13:49:56.603 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T13:49:56.629 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T13:49:56.629 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T13:49:56.646 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T13:49:56.695 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T13:49:56.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T13:49:56.716 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T13:49:57.149 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T13:49:57.160 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T13:49:57.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T13:49:57.185 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T13:49:57.212 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T13:49:57.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T13:49:57.229 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T13:49:57.250 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-13T13:49:57.275 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-13T13:49:57.293 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T13:49:57.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:49:57.321 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T13:49:57.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T13:49:57.347 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T13:49:57.357 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T13:49:57.358 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647/remote/smithi043 2024-09-13T13:49:57.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T13:49:57.426 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-13_12:30:34-orch:cephadm-wip-guits-main-2024-09-13-0840-distro-default-smithi/7903647/remote/smithi189 2024-09-13T13:49:57.427 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T13:49:57.466 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T13:49:57.466 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T13:49:57.469 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T13:49:57.514 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T13:49:57.530 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T13:49:57.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T13:49:57.540 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T13:49:57.541 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T13:49:57.543 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T13:49:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 13:49 /home/ubuntu/cephtest 2024-09-13T13:49:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 13:49 /home/ubuntu/cephtest 2024-09-13T13:49:57.571 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T13:49:57.581 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T13:49:57.602 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} duration: 2015.650958776474 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-13T13:49:57.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T13:49:57.697 INFO:teuthology.run:pass